-- Logs begin at Mon 2022-09-12 14:56:57 UTC, end at Mon 2022-09-12 16:28:59 UTC. -- Sep 12 15:22:28.039204 dv-cinder-851996-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 12 15:22:28.047838 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 12 15:22:28 2022] *** Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: nodename: dv-cinder-851996-10-iscsi Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: machine: x86_64 Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: clock source: unix Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: pcre jit disabled Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: detected number of CPU cores: 4 Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: current working directory: / Sep 12 15:22:28.049917 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: detected binary path: /usr/bin/uwsgi-core Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: your processes number limit is 47680 Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: your memory page size is 4096 bytes Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: detected max file descriptor number: 2048 Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: lock engine: pthread robust mutexes Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: thunder lock: enabled Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 12 15:22:28.050569 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 12 15:22:28.090270 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: Python main interpreter initialized at 0x5570e3de79c0 Sep 12 15:22:28.090270 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: python threads support enabled Sep 12 15:22:28.090270 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: your server socket listen backlog is limited to 100 connections Sep 12 15:22:28.090270 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: your mercy for graceful operations on workers is 90 seconds Sep 12 15:22:28.090610 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: mapped 403077 bytes (393 KB) for 2 cores Sep 12 15:22:28.090687 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: *** Operational MODE: preforking *** Sep 12 15:22:28.090941 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: *** uWSGI is running in multiple interpreter mode *** Sep 12 15:22:28.090941 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: spawned uWSGI master process (pid: 81387) Sep 12 15:22:28.091127 dv-cinder-851996-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 12 15:22:28.094336 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: spawned uWSGI worker 1 (pid: 81399, cores: 1) Sep 12 15:22:28.094336 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: spawned uWSGI worker 2 (pid: 81400, cores: 1) Sep 12 15:22:28.094336 dv-cinder-851996-10-iscsi devstack@keystone.service[81387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 12 15:22:32.376734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 15:22:32.376734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 12 15:22:32.377405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 15:22:32.377405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 12 15:22:32.377523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 15:22:32.377523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 12 15:22:32.377955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 15:22:32.377955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 12 15:22:32.427298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 15:22:32.427488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 15:22:32.427668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] command line args: [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 15:22:32.428536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 15:22:32.428717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ================================================================================ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 15:22:32.428975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.429223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_dir = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.429420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.429609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_source = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.429834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] debug = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.430039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_config_append = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_dir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.431953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.432193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.432443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.432690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.432936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.433020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 15:22:32.433116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.433194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 15:22:32.433317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.433443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] command line args: [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 15:22:32.433568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.433680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 15:22:32.433809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.433933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ================================================================================ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 15:22:32.434045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_dir = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_source = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.434975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] debug = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.435898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] publish_errors = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_config_append = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_dir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.436926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] standard_threads = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.437881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_journal = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_json = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.438914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_stderr = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_syslog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.439729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.439841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.439959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.440077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.440195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.440307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.440426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.440538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.440650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.440763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.440763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.440979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.441137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.441253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] publish_errors = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.441370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.441483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.441605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.441734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.441847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.441958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.442079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.442192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.442315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.442429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.442552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.442671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] standard_threads = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.442671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.442881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.443004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.443118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.443229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.443349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.443462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_journal = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.443573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.443696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_json = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.443696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_stderr = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.443914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.444026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_syslog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.444136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.444247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 15:22:32.444366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.444479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.444839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.444952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.445915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.446922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.447965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.448898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.449928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.450961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.451898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.452993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.453931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.454939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.455982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.456993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.457935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.458953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.459937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.460974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.461912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.462950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.463971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.external = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.464997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.password = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.465916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.token = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.466944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.467898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.468942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.469929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.external = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.470962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.password = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.token = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.471912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.472947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.473999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.474112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.474226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.474339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.474574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.475901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.476946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.477944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.478994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.479912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.480957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.481892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.482945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.483984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.484998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.485930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.486972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.487887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.488920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.489978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.490091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.490204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.490877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.491805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.492945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.493958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.494881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.495889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.496984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.497955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.498984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.499911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.500893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.501957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.502975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.503895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.504942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.505947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.506980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.507981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.508910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.509957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.510896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.511936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.512968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.513998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.514901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 15:22:32.515251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.515893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.516006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.516219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.516461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.516703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.516943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.517186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.517435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.517677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.517947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.518189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.518441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.518681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.518930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.519166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.519410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.519649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 15:22:32.519821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 15:22:32.543537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.543770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.543996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.544981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.545219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.545450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.552348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.554415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.581057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.581285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.583055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.583295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.634816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.635023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.684142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.684377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.684463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.684682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.840974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.841120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.841246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.841329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.863958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.864958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.865089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.865170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.865839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.865921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.866058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.877021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.877198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.890978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.891737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 15:22:32.908262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.908474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 15:22:32.908623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.908820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.908906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 15:22:32.909025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 15:22:32.909176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.909306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.909389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 15:22:32.909573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.909662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.909861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.909944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.910125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.910209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.910439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.919765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.920948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.927602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.928598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.931063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.931352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.931546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.931628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.931836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.931959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.932964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.933892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.934089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.934363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.957168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.957913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.958666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.958951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.959414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.959707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.963641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.963931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.964222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.964400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.964691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.964979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.970598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.970886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.971390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.971685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.976536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.976838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.977131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.977338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.977421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.977636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.977741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.977950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.978789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.979065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.979339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.994390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.994677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.995340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.995626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:32.998478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:32.999511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.001641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.001968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.002787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.003089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.006714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.007864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.009954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.010997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.013179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.013485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.013798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.014877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.015171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.015457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.015762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.030440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.030732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.031005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.031273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.031595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.031888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.032162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.032428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.038186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.039321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.039715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.040875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.041245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.041528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.042439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.042719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.044301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.044596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.044898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.045187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.045414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.045580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.045732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.046026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.046326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.046718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.057257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.057544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.057858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.058147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.058414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.058532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.058706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.058824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.058997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.059108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.059287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.059677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.059981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.060270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.075228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.076443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.078157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.079393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.081042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.082313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.082523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.083750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.084011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.084297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.084695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.085367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.085656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.086081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.091086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.092465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.092548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.093948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.095433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.095832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.096149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.096432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.096849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.097253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.097546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.097858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.104765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.105059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.105346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.106225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.106525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.106812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.112217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.112504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.112789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.113070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.113349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.113625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.113797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 15:22:33.113930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.114859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.115050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.115139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.115962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.116146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.116227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.116460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.116584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.116749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.117036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.117313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.117585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.117893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 15:22:33.150307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5570e3de79c0 pid: 81399 (default app) Sep 12 15:22:33.151731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5570e3de79c0 pid: 81400 (default app) Sep 12 15:22:33.153493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725bfe-1bb5-4c34-a81a-f9d769c3d0a3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:33.153810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725bfe-1bb5-4c34-a81a-f9d769c3d0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:33.154087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f725bfe-1bb5-4c34-a81a-f9d769c3d0a3 None None] PATH_INFO: `/v3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:33.158239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Sep 12 15:22:33 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:39.918074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:39.918593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:39.918699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:39.922937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:22:39.928151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:39.928650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:39.928745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:39.998774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: INFO dbcounter [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] Registered counter for database keystone Sep 12 15:22:40.017846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG oslo_db.sqlalchemy.engines [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 15:22:40.024502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writer thread running {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 15:22:40.201057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 15:22:40.201362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 15:22:40.259694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:40.261298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:40.391130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 15:22:40.831700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:40.901781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: INFO dbcounter [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] Registered counter for database keystone Sep 12 15:22:40.922809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG oslo_db.sqlalchemy.engines [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 15:22:40.929537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writer thread running {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 15:22:41.046926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13c5e-4c72-45ca-8b20-12ca5ef5b301 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:41.051349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:41.051971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:41.052262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:41.053058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:41.061192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:41.112179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 15:22:41.112483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92257fdb-d968-4b3e-a39e-053496de8f8a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:41.122354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Sep 12 15:22:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 15:22:41.128308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:41.145336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07019-0130-4a87-b73b-ebbbb990c02a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:41.149736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:41.150841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:41.151129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:41.151402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] PATH_INFO: `/v3/projects/45e559d5550544c28dd39b8ed456ebde` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:41.152985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] RBAC: Authorizing `identity:get_project(project_id=45e559d5550544c28dd39b8ed456ebde)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:41.205357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 15:22:41.205727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa918ed-e0b2-44ff-ab70-6ac9c1b9844d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:41.210262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Sep 12 15:22:41 2022] GET /identity/v3/projects/45e559d5550544c28dd39b8ed456ebde => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:43.326600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be35f1b-1b7d-476d-a818-e5af190dc557 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:43.327192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be35f1b-1b7d-476d-a818-e5af190dc557 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:43.327192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be35f1b-1b7d-476d-a818-e5af190dc557 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:43.331278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:43.336730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:43.337165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:43.337284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:43.357415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:43.358965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:43.367314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 12 15:22:43.630253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be35f1b-1b7d-476d-a818-e5af190dc557 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:43.642266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be35f1b-1b7d-476d-a818-e5af190dc557 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:43.646555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:43.647152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:43.647440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:43.647721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:43.655345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:43.657138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d129-34b3-414c-8bcd-9a897eb513cb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:43.666410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Sep 12 15:22:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 15:22:43.671442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:43.681035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee122c-a4d3-4b1b-9a25-3833211e7d1c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:43.685273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:43.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:43.686203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:43.686477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:43.687781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:43.689599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd394fff-057d-426d-addb-4329747efe98 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:43.691184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Sep 12 15:22:43 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:45.893164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb567e3-a291-42c5-8b59-9f284395a793 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:45.893688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb567e3-a291-42c5-8b59-9f284395a793 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:45.893833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb567e3-a291-42c5-8b59-9f284395a793 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:45.895084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:45.900754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:45.901133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:45.901420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:45.919234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:45.920410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:45.922676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242096-af5f-409b-8312-6de8b40a6768 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:45.922968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242096-af5f-409b-8312-6de8b40a6768 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:45.923250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e242096-af5f-409b-8312-6de8b40a6768 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:45.924553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:22:45.929810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:45.930016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:45.930302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:45.973557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 15:22:45.973979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 15:22:46.006525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:46.023308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:46.024902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:46.122644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:22:46.277500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.303789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.388989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d6e82-315c-4fa2-9849-355ce8fd341b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.389653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.423853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.424487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.424767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.425042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.436736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:22:46.446416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:22:46.447605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:22:46.448419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:22:46.453100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.492638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf530d-6a06-4df1-896f-130f987be1b2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.494909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.531240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.532033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.532319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.532593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.534487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.553122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95257091-70eb-4e5f-88f5-18ca47725700 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.586842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.587449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.587730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.588002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.588664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:46.588965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:46.590388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:46.600639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:46.602286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:46.602465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:46.607836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.700620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:22:46.704589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33be82-5787-4411-aed0-0bb8e580f44a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.706156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.708749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.738259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.738895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.739181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.739454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.741159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:46.742655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:46.744213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:46.749465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.764795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.804888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939577-b1e2-4a14-9ab9-712e748831e9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.839309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.840163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.840447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.840726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.851242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400b197-e5a5-45e5-a7db-8d469403aa7f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.859260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:46.860736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:22:46.879132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:22:46.879997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:22:46.880776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:22:46.880776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:22:46.880776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:22:46.880776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:22:46.885336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:46.886006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:46.886273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.886380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:46.886546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:46.887640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:46.889101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:46.890670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:46.896233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:46.942413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.978869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:46.988147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f25be-0dc3-407a-a641-bf07669399cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:46.998663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba0e95-cef5-425d-8106-25dec01e55b4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:47.020375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:47.021505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:47.021817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:47.022102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:47.022747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:47.023052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:47.024439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086b2cd-28e0-42a7-ba22-752e2763811a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:47.031920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:47.032530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:47.032815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:47.033092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:47.033860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.domain.id=default, user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:47.034176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:47.035609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb74-3003-4023-a3e8-20ecdcb4a855 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:47.042162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:47.044678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Sep 12 15:22:46 2022] GET /identity/v3/role_assignments?user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c&scope.domain.id=default&role.id=c939df765d074277be1dc85b4ec66f05 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:49.372571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8148f-db45-48e0-8b68-7ecbc904b550 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:49.372972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8148f-db45-48e0-8b68-7ecbc904b550 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:49.373324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8148f-db45-48e0-8b68-7ecbc904b550 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:49.375005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:49.380391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb10f-0ac4-485a-a8ce-3522b7e2f780 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:49.381264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb10f-0ac4-485a-a8ce-3522b7e2f780 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:49.381772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eeb10f-0ac4-485a-a8ce-3522b7e2f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:49.401415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-37eeb10f-0ac4-485a-a8ce-3522b7e2f780 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:49.402981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-37eeb10f-0ac4-485a-a8ce-3522b7e2f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:49.495569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:22:49.740912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2b66-ab4b-4d44-b0f7-df41f6439605 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:49.741540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2b66-ab4b-4d44-b0f7-df41f6439605 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:49.742046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2b66-ab4b-4d44-b0f7-df41f6439605 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:49.743828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:49.749394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7828-52a7-456b-80ba-d427ead63f25 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:49.749937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7828-52a7-456b-80ba-d427ead63f25 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:49.750426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c7828-52a7-456b-80ba-d427ead63f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:49.783363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bb2c7828-52a7-456b-80ba-d427ead63f25 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:49.784889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bb2c7828-52a7-456b-80ba-d427ead63f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:49.857037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:22:50.049959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2b66-ab4b-4d44-b0f7-df41f6439605 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.174749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2b66-ab4b-4d44-b0f7-df41f6439605 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.201153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:50.223454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.224683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.224971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.225250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.233867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70710f-199c-4325-a544-3e1303045c65 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.234121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70710f-199c-4325-a544-3e1303045c65 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.234323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70710f-199c-4325-a544-3e1303045c65 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.235981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:50.241010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.241298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.241578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.252069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:50.265551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cd60e9c0-d826-4e3d-abd9-c763efc4e34b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:22:50.275357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:22:50.276173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:50.276940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:22:50.278065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:50.278721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:22:50.279457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:50.279990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5d9ff-05dd-43e4-956d-76c2eccff497 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.280280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5d9ff-05dd-43e4-956d-76c2eccff497 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.280556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5d9ff-05dd-43e4-956d-76c2eccff497 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.282337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 14/28] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:50.283971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5d9ff-05dd-43e4-956d-76c2eccff497 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.357494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 15/29] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:50.358324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.449623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5d9ff-05dd-43e4-956d-76c2eccff497 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.512432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.513069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.513347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.513618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.532544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:50.534064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:50.551054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0924b-9ef9-448f-a520-900acdeb64fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-879d5a40-fef7-46c6-a8dd-95b7f7c0e27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:22:50.568652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:22:50.569883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:22:50.570666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:22:50.570666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:22:50.570666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:22:50.570666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 15/30] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:22:50.571599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.571599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.571763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.602060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.602688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.602971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.603247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.603927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:50.604232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:50.606908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:50.613028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:50.614217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:50.620016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:50.623469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 16/31] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:50.624676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.662885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:50.721826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 16/32] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:50.723074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.760254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf373a8a-f0ff-4c0f-8b61-63b525c10ec8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.795516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.796215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.796507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.796787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.797435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:50.797769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:50.799809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:50.818691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 17/33] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:50.824169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.883569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:50.910150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3770e9-4152-44c3-ba54-17f5cc60141e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.954891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc83b75-bc91-4847-844f-b9a05aeba4e0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:50.982858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.983631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.983919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.984200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.990962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:50.992510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:50.995339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 17/34] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:22:50 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:50.995870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:50.996247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:50.996993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:50.997550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:50.998030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:50.998784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:51.000270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:51.058216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:51.132353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 18/35] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Sep 12 15:22:50 2022] POST /identity/v3/roles => generated 225 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:22:51.132353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:51.144839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d854817-4062-479b-8af1-ab0de379ba03 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:51.189506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:51.190758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.191366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.191858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.192795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:51.195020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb96420-0094-46b5-bb22-7c57e0dfea64 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:51.251693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8c00-7f9a-4012-8b24-c13cada722af None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:51.271252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:51.295508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:51.296197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.296486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.296760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.298258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:51.299910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:51.302547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 19/36] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:51.303303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:51.362544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:22:50 2022] POST /identity/v3/projects => generated 301 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:22:51.367269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e2a1-9ab1-451c-8eb5-85385958278f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.367269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e2a1-9ab1-451c-8eb5-85385958278f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.374022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9e2a1-9ab1-451c-8eb5-85385958278f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.374022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 19/38] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:51.374022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9e2a1-9ab1-451c-8eb5-85385958278f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:51.454611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d10c43-c70e-4cdc-a326-72de2f8f1681 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:51.481726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:51.499888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:51.500532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.500817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.501124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.501854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:51.503287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c716072-aad9-46cf-bc6b-9cabb9e152f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:51.510006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9e2a1-9ab1-451c-8eb5-85385958278f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:51.518830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 20/39] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Sep 12 15:22:51 2022] POST /identity/v3/roles => generated 219 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:22:51.520474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.521322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.521609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.566616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:51.568451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.568977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.569747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] PATH_INFO: `/v3/domains/default/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.573069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:51.575276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e91de-2a85-46ea-919e-d33de1b24abc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:51.595758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 20/40] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Sep 12 15:22:51 2022] PUT /identity/v3/domains/default/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:22:51.597744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dcab6-340c-4f83-b88d-800bbde08b60 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.598290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dcab6-340c-4f83-b88d-800bbde08b60 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.598902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dcab6-340c-4f83-b88d-800bbde08b60 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.600764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 21/41] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:51.608071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:51.608747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:51.609228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:51.636250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:51.637950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:51.660804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:51.662675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:51.730037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:51.733212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:51.759650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 21/43] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:52.014148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:52.028071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:52.085464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:52.123562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b714f-ffe0-4274-a622-35b8f3829c73 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:52.132646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a78fb-d871-4681-be1f-fd891e0172b9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:52.158793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:52.159435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:52.159746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:52.159987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:52.161077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:52.163118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:52.164913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 23/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:52.167983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:52.168609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:52.168899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:52.169190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:52.169912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:52.170187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:52.171325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfc7bd-252a-47a2-aafe-5a562b47ca20 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:52.187338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 22/45] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Sep 12 15:22:52 2022] POST /identity/v3/roles => generated 223 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:22:52.278033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93edc1-ca55-428f-939b-07c07c91e37d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:52.314152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:52.314785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:52.315069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:52.315392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:52.316009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:52.317514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da94da-40e4-4fae-8a79-bfcb4d9ab987 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:52.323801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:52.350730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:22:52 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:22:53.839386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be81c0-e00c-479b-b6a3-a4c0ed790cfc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:53.839923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be81c0-e00c-479b-b6a3-a4c0ed790cfc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:53.840006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be81c0-e00c-479b-b6a3-a4c0ed790cfc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:53.841301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:53.846569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:53.846943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:53.847136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:53.865270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:53.866486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:53.931234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:22:54.212426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be81c0-e00c-479b-b6a3-a4c0ed790cfc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.247535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:54.315783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be81c0-e00c-479b-b6a3-a4c0ed790cfc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:54.350833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:54.351474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.351760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.352038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.363511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-be0a092c-ffc8-4290-aa40-f7e63f8a6125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:22:54.368696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:22:54.369821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:22:54.370595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 24/49] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:22:54.376029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.476985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c0063-62e0-4d1d-bab7-a9fa85dae6e6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:54.494096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:54.494692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6318167-8767-40b6-b54b-15332ec539d6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.494999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6318167-8767-40b6-b54b-15332ec539d6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.495288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6318167-8767-40b6-b54b-15332ec539d6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.496957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 25/50] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:22:54.502310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.502753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.503043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.512253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:54.512861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.513142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.513416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.514118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:54.514424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:54.515814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:54.521384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:54.522634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:54.527366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 26/51] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:54.533254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.590321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 26/52] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:22:54.629290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9195e-96ae-40b8-82bd-4b2fee40c81d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:54.662391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:54.663023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.663309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.663585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.664706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:54.666191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:54.667742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 27/53] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:54.673302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.702472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:54.776089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87c20-c9ec-479f-8846-ea4baea68498 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:54.810120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:54.810751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:54.811036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:54.811316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:54.812426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:54.813908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:54.815628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:54.822336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.864074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:54.962133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:54.972115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51d9fd-4981-4613-8283-47e85b968800 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:54.974264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:55.009004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:55.009639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:55.009950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:55.010230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:55.011344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:55.012794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:55.014393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 28/55] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:22:55.019879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:55.026411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade950cb-7135-4436-ab09-d9c7476af5a8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:55.059646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:55.060251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:55.060534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:55.060810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:55.061532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.domain.id=default, user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:22:55.061877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:55.063322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee3c85-0ef9-48d3-b562-3a4b1ba2b5de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:55.071587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 28/56] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Sep 12 15:22:54 2022] GET /identity/v3/role_assignments?user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c&scope.domain.id=default&role.id=c939df765d074277be1dc85b4ec66f05 => generated 530 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:55.122245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325850c-64b3-48a8-948c-a82e55008a87 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:55.160893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:55.162128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:55.162418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:55.162697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:55.163368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:55.164843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428839a-b6dd-46a1-a380-2372d04fb69b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:55.194896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 29/57] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Sep 12 15:22:55 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:22:57.351284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa58b0-62ab-477c-ab70-6a03c706d30b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:57.351804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa58b0-62ab-477c-ab70-6a03c706d30b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:57.351887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa58b0-62ab-477c-ab70-6a03c706d30b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:57.353176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:22:57.358417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:57.358866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:57.358988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:57.377896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:22:57.379102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:22:57.386406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:57.446296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:22:57.714296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa58b0-62ab-477c-ab70-6a03c706d30b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:57.811242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa58b0-62ab-477c-ab70-6a03c706d30b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:57.824415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:57.844343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:57.844961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:57.845246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:57.845523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:57.846656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:57.848107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3158a23-8ef6-4920-83f5-e35077e03e14 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:57.850213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:22:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:22:57.855496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:22:57.952255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c21417-0d36-4679-9969-eed598e7f87e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:22:57.986926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:22:57.987483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:22:57.987770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:22:57.988050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:22:57.988734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:22:57.990777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b1a1-0982-4b01-8498-7c3262995482 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:22:58.011236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:22:58.016950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:22:57 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:01.215161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93be3-d244-4f75-a2af-e73452985ab5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.215674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93be3-d244-4f75-a2af-e73452985ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.215781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93be3-d244-4f75-a2af-e73452985ab5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.217199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:01.234503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ee98-947b-4464-b080-c3fd797b21ec None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.235341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ee98-947b-4464-b080-c3fd797b21ec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.235823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6ee98-947b-4464-b080-c3fd797b21ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.262960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-87c6ee98-947b-4464-b080-c3fd797b21ec None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:01.264527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-87c6ee98-947b-4464-b080-c3fd797b21ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:01.363997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 32/63] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:01.403426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4f9ac-81e5-437a-a284-cb7af00443f5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.403744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4f9ac-81e5-437a-a284-cb7af00443f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.403995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4f9ac-81e5-437a-a284-cb7af00443f5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.405575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 32/64] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:01.411200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.412010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.412295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.431537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:01.432741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:01.505830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59f77-d509-4b2e-9d2b-e31253f8fe7b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.505830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59f77-d509-4b2e-9d2b-e31253f8fe7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.505830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59f77-d509-4b2e-9d2b-e31253f8fe7b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.507033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 33/65] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:01.512744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.513103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.513596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.533624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 33/66] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:01.539385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:01.540603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:01.615133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 34/67] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:01.685456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:01.747865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:01.808689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3058f0-43f0-4a04-b803-731e97671630 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:01.850725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:01.851375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:01.851662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:01.851941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:01.862920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:01.868179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:01.869280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:01.870270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 34/68] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:01.875883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:01.963003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:01.995713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f39c6-b82c-4ed4-b8ec-ada3bcb5f335 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.008850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.037439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.038199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.038483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.038819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.039490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:02.039845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.041243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.050190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 35/69] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.055040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.106351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5ef9e-de24-4a21-9337-5118c49ab29d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.127226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.146133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.146993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.147281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.147559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.159084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-95334102-05f3-48b9-9e6c-677944ff6aab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:02.164108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:02.164936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:02.165696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 35/70] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:02.167282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cffd2-4c9c-42b1-a598-884532b1436d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.167282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cffd2-4c9c-42b1-a598-884532b1436d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.167282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cffd2-4c9c-42b1-a598-884532b1436d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.170175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 36/71] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:02.170175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cffd2-4c9c-42b1-a598-884532b1436d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.181793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4117-b151-4787-9d7a-5f92a608f91f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.218234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.218905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.219141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.219422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.229381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:02.234373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:02.235191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:02.235930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 36/72] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:02.238563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.269995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.276798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cffd2-4c9c-42b1-a598-884532b1436d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.312818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.313459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.313777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.314062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.328046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.329551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbab5f-7bd5-438b-8973-757cb6b21743 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.331194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 37/73] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.335842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.335842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.335842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.338651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d80e-ad7f-42fa-a51c-062cf4fd1eaa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.350441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:02.351650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:02.375841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.376453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.376736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.377012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.377749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:02.378060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.379504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b525fe-dec5-4b39-bc23-9ef4af8ce95d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.383422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 37/74] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d6c3c-b49b-441b-9ca1-8a00fe4ea77b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d6c3c-b49b-441b-9ca1-8a00fe4ea77b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d6c3c-b49b-441b-9ca1-8a00fe4ea77b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 38/75] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:02.392403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d6c3c-b49b-441b-9ca1-8a00fe4ea77b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.428808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 38/76] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:02.429641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.488470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.497993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d6c3c-b49b-441b-9ca1-8a00fe4ea77b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.533508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.534291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.534646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.534991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.535653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd0f09-a160-4316-af0b-db5ebd684483 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.536313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:02.536696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.538368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54f505-84fe-4648-8f6a-736a3aac8cc2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.548584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 39/77] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.549744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.550036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.550308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.568174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:02.569617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:02.577478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.578291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.579062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.579569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.615961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.643286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 40/78] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:02.644516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.666095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.668357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.737640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.758419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cae18-fca1-409e-a2da-efb6ec9c2c23 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.779940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 39/79] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:02.780555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.799588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.800256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.800540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.800814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.801997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.803457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.809521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 41/80] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.809521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.854383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:02.888241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cdfb5-42d6-4109-8712-3f857a118519 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.914638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0813ea3-4436-4134-9c90-050688e2e499 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:02.925270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.925928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.926211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.926519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.927651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:02.929126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:02.930741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:02.931498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.949102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:02.949747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:02.950033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:02.950306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:02.961023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:02.965946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:02.966951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:02.967676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 42/82] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:02.972424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:02.988025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.030789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8fe4-4650-4b31-a579-35ac191d91f9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.063971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.064612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.064901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.065184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.066327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.068552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.070204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 41/83] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.070960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.071421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01b6bc-ebe8-438c-8c7c-dc44ce452442 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.073488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.105738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.106337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.106627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.106907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.170037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b390-9074-4e10-8377-377cf9259304 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.176607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.178124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.191881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.203504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.204141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.204425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.204702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.205840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.207294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.208930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 42/84] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.209690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.277317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.280923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 43/85] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:03.281619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.310499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953bd1-fc20-43cb-bfc7-f41eeb4858cc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.343874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.344484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.344796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.345071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.385089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0d1c2-4860-48dd-8aac-0be3ca7c66ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.398626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.415177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.416676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.418772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.419382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.419671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.419949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.420658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:03.420918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.422345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.430842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 44/86] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.434458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.484608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.515832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 43/87] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:03.516627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.534094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314b6ef-8e8c-4865-80e2-3422c4063594 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.567799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.568406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.568687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.568963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.569977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:03.570291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.571750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.576911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 45/88] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.577788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.618945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ccc92-dde7-400c-b338-0b1ac0984e5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.635418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.658632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.659258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.659544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.659823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.660510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.662081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.685052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 44/89] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:03 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:03.685752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.707085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a4136-7a60-4e7f-a9e3-72ce38d57f2d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.728049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.760186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.760855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.761145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.761427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.762788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.764979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.766949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 46/90] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.772668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.822906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868fdf-851c-471f-9e54-6157cddbe05c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.865731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.866567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.866826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.867105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.868759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.870266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.872588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 45/91] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.873266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.889439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c6ad-bdf5-4674-93f2-5f5335caa2fa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:03.900747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.945445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:03.946381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:03.946901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:03.947103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:03.948313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:03.949834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:03.951641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 47/92] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:03.952240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:03.990352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:03.998632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98596650-190d-4327-9eba-5df4f5fc18f9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:04.048335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:04.048966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:04.049256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:04.049541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:04.050368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:04.050691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:04.052743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:04.060491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 46/93] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:04.061357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:04.092809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df598d4-1e33-4c13-8fbc-db82dadfc143 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:04.142886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:04.143577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:04.143870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:04.144147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:04.198242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:04.228314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:04.230125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:04.238117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e6a8-5aa9-4e5e-8138-6e54ded79d27 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:04.309737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:04.310520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:04.310755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:04.311040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:04.311835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:04.312154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:04.313634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:04.323977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 47/94] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:04 2022] GET /identity/v3/role_assignments?user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:04.404528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 48/95] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:04.410507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:04.445383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:04.561475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e1af-c1ef-4cbd-adf4-eb413fbb6f81 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:04.599191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:04.599857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:04.600141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:04.600421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] PATH_INFO: `/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:04.601632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] RBAC: Authorizing `identity:get_project(project_id=8ed0e5be52c64ebd816a53beb4a8a9ce)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:04.603141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114e76e-cd63-408a-a071-3222d329e2af None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:04.604812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:04 2022] GET /identity/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:04.610928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:04.757867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:04.786356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e058da1-2b9e-40d1-870e-a3a9546d61c0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:04.831772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:04.832320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:04.832634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:04.833019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:04.833849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=8ed0e5be52c64ebd816a53beb4a8a9ce, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:04.834169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:04.835691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a870eb-f7f1-460e-af5a-6271655190dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:04.846049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:04 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=8ed0e5be52c64ebd816a53beb4a8a9ce&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:06.788032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c6943-c955-49cf-875d-59a162a4c50e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:06.788890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c6943-c955-49cf-875d-59a162a4c50e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:06.789316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c6943-c955-49cf-875d-59a162a4c50e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:06.791170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 49/98] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:06.796234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bcde0-5d3d-4e4d-b771-d9a50512b347 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:06.796694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bcde0-5d3d-4e4d-b771-d9a50512b347 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:06.796783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bcde0-5d3d-4e4d-b771-d9a50512b347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:06.823381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-294bcde0-5d3d-4e4d-b771-d9a50512b347 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:06.824613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-294bcde0-5d3d-4e4d-b771-d9a50512b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:06.879249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b225d9-f40d-4ba7-87e4-08fa7077e1e4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:06.879923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b225d9-f40d-4ba7-87e4-08fa7077e1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:06.880403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b225d9-f40d-4ba7-87e4-08fa7077e1e4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:06.883878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 50/99] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:06.889108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24580f8-25f7-4b54-abbe-e0d72d5373d0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:06.890582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24580f8-25f7-4b54-abbe-e0d72d5373d0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:06.891097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24580f8-25f7-4b54-abbe-e0d72d5373d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:06.919943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c24580f8-25f7-4b54-abbe-e0d72d5373d0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:06.921407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 50/100] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:06.926153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c24580f8-25f7-4b54-abbe-e0d72d5373d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:07.003902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 51/101] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:07.144757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d547-28c3-4050-b5bb-183f62845b51 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.145956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d547-28c3-4050-b5bb-183f62845b51 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.146436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2d547-28c3-4050-b5bb-183f62845b51 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.147991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 51/102] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:07.155261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0667c-4136-451c-ae73-2fe586f42c8e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.155561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0667c-4136-451c-ae73-2fe586f42c8e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.157198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0667c-4136-451c-ae73-2fe586f42c8e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.157198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 52/103] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:07.158774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.159227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.159647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.180368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:07.186103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.186572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.187002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.191955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:07.217912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:07.219332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:07.254046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:07.318110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:07.326908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 52/104] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 12 15:23:07.339916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 53/105] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:07.396340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.484241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.540782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b9970-10a0-4fd8-806c-a56277628658 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:07.608563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:07.609446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.609927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.611299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.631905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:07.638403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:07.639431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:07.640372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:07.642345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 53/106] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:07.643253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.680742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b729-301b-43aa-a81f-3ae3f46ca84c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:07.706087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:07.737190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:07.738243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.738905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.739472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.748128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ecfdd-23ea-4782-b226-d282edd60084 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:07.750963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:07.757010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:07.757853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:07.758818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:07.760981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 54/107] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:07.761986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.775526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:07.786218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:07.786871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.787160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.787442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.788567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:07.790237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:07.791971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 54/108] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:07.792883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.894623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acbe08-b7a4-47bf-ad47-d825ab79cd7b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:07.913668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bd1c-d808-4015-8392-2a3b1e2a6b8e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:07.944569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:07.944728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:07.946489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.946872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.947313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.948396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:07.949732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:07.951755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:07.954753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:07.955550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:07.956123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:07.956534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:07.962409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 55/109] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:07.963621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:07.969513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:07.975204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:07.976079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:07.976868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 55/110] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:07.977507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.061027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.095220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ae3d2-dbb7-4ebb-ad76-e7ce4e3797b9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.098950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e1b9-6186-4fb9-88a0-c2abcdd6dcb9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.133214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.134232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.134437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.135008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.135233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.135329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.135571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.135974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.136243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:08.136557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.137215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.137979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.139111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.146711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 56/111] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.147510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.169766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 56/112] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:23:07 2022] POST /identity/v3/projects => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:08.170669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.226811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.274108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd518-dd50-4e7b-8272-04e038dadd2c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.297539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dc61c-7fbf-41f7-b295-084eef69448e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.314471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.315425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.315897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.316356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.318410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.320243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80018b6-384d-4053-847d-303d3ba41f34 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.322132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 57/113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.323235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c226-38e6-4345-919b-3013b48c3d3a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.323534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c226-38e6-4345-919b-3013b48c3d3a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.323815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c226-38e6-4345-919b-3013b48c3d3a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.329292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 58/114] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:08.329292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102c226-38e6-4345-919b-3013b48c3d3a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.333237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.345927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.346572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.346903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.347130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.348167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:08.348493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.349947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529a4a-005c-4f45-90ab-bb9a4b496fa4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.363560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 57/115] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.363752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ac13-548b-4f20-bde9-feb17ff756d6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.363983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ac13-548b-4f20-bde9-feb17ff756d6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.364267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ac13-548b-4f20-bde9-feb17ff756d6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.365551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 58/116] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:08.370352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ac13-548b-4f20-bde9-feb17ff756d6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.456397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102c226-38e6-4345-919b-3013b48c3d3a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.493847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.494412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.494701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.494984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] PATH_INFO: `/v3/users/9ee5beb6969e4bab9f23c826894f841a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.495801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ac13-548b-4f20-bde9-feb17ff756d6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.497430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.510889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] RBAC: Authorizing `identity:get_user(user_id=9ee5beb6969e4bab9f23c826894f841a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.513367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb468c-ad63-4c2e-be98-47f8c573a753 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.520748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 59/117] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/users/9ee5beb6969e4bab9f23c826894f841a => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.520748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf12f3a-231d-400d-9119-c3b38e373797 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.520748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf12f3a-231d-400d-9119-c3b38e373797 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.520748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf12f3a-231d-400d-9119-c3b38e373797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.539226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0cf12f3a-231d-400d-9119-c3b38e373797 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:08.540132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.540750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.541036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.541202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0cf12f3a-231d-400d-9119-c3b38e373797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:08.541306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.542479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.543906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.545736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 59/118] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.546321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.606352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.618709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 60/119] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:08.619754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.620048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.620331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.646995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:08.648656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:08.667436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfd26-bb3c-4074-989c-81f0fd186145 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.712926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.713573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.713896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.714180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.716045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.717508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.719204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 60/120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:08.725000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.743496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 61/121] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:08.751237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:08.765425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.829159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692b4a-935c-4051-9194-bb507a4072a4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.874475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.875463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.875954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.876429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.914854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac2d64-8437-4201-ac01-b8163842838c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:08.945827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:08.958615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.961277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.964724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:08.965155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:08.965539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:08.965945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/users/fe943a904f9347cdbc2ea3c7493a9421/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:08.967998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, user_id=fe943a904f9347cdbc2ea3c7493a9421, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:08.969556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:08.987914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 62/122] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:08 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/users/fe943a904f9347cdbc2ea3c7493a9421/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:08.990361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.028573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.154266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 61/123] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:09.155618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.172910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836fdbf-dbb7-4f89-8331-7b8930562b37 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.214994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.216664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.217188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.217665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.219088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:09.220813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:09.222702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 63/124] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:08 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:09.228444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.281180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b97e49-61d8-44c7-b699-9c24126f8860 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.300114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.320068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.320763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.321057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.321335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.328765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.333453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:09.339634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:09.340664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:09.341609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:09.344387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 62/125] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:09.344961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.376956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8533-9d77-4bab-a7ad-2d56004ad509 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.412657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.413410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.413720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.414025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.424811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:09.430845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:09.431718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:09.432467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 64/126] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:09.433091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.501189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de822c0-1236-494f-9cf1-108700bcf085 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.544323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2a99-0ba5-4b74-8334-5dccfa325d17 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.548933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.549812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.550290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.550738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.552164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:09.554380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:09.555300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.558209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 63/127] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:09.559213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.592465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.593675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.594158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.594943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/users/fe943a904f9347cdbc2ea3c7493a9421/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.598285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, user_id=fe943a904f9347cdbc2ea3c7493a9421, role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:09.600512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:09.619725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 65/128] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:09 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/users/fe943a904f9347cdbc2ea3c7493a9421/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:09.620859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.655834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.723437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1cf9-4c5a-4386-9249-4b58305cbfde None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.761400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c01c0-75f5-4b29-bee4-777d9e8a8046 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:09.810478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.811014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:09.811197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.811584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.811930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.812383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:09.812759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:09.813023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:09.813119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:09.813540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:09.814391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:09.814738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:09.815066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:09.817613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:09.833430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 66/129] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:09.834729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.837533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 64/130] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:09.838491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:09.886208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:09.982518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49600f3-a00e-47d7-a547-b92408fc187d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.005972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7dd10-aa26-4002-a9e0-50b9934b0094 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.032808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.034071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.034793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.035260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.036071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:10.037697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=9ee5beb6969e4bab9f23c826894f841a) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:10.038278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.040762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.045464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.046333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.046681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.047008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.048263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.048666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/role_assignments?user.id=9ee5beb6969e4bab9f23c826894f841a&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:10.049651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:10.054282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.058548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:09 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:10.081441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:10.279888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76314ead-1000-469b-bc22-afe32b197d34 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.306379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:10.326513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348b4e8-299c-483b-a14c-46fc86326c02 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.329336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.330229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.330653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.331067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.332622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.335445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.343625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 66/133] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:10 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:10.352807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:10.422566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.423225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.423507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.423782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.428699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.430504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.436412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 68/134] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:10 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:10.437392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:10.497401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:10.547430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434af5-7c99-4a4b-90d2-df0e5aca733c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.586317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.587866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.588628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.589118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.591428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.593792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced6f47-ac9f-45ef-9602-ef7db91cad2e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.599750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843c63-5f75-4a4d-baf8-d918e4edbd52 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.616304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 67/135] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:10 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:23:10.617945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7195-65c7-407f-a355-31063e8c5310 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.618478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7195-65c7-407f-a355-31063e8c5310 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.625738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7195-65c7-407f-a355-31063e8c5310 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.629219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 68/136] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:10.635257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.635790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.636272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.644796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.645477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.645788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.646067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] PATH_INFO: `/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.647426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] RBAC: Authorizing `identity:get_project(project_id=8ed0e5be52c64ebd816a53beb4a8a9ce)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.648998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.650922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 69/137] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:10 2022] GET /identity/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:10.659004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:10.673534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:10.674809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:10.725421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:10.809740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53a665-805c-4f10-b179-c14b22190746 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:10.811380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 69/138] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:10.845780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:10.846384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:10.846669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:10.846949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] PATH_INFO: `/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce/users/fe943a904f9347cdbc2ea3c7493a9421/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:10.849755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ed0e5be52c64ebd816a53beb4a8a9ce, user_id=fe943a904f9347cdbc2ea3c7493a9421, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:10.851505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:10.874545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 70/139] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:10 2022] PUT /identity/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce/users/fe943a904f9347cdbc2ea3c7493a9421/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:23:11.395996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:11.468018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:11.518937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313d51-85a0-4f67-8356-f378b9d2c47e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:11.561332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:11.562011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:11.562299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:11.562577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:11.563708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:11.565166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a896bc-8096-451a-be75-907f4428a1c9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:11.567307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 70/140] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:11.574774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:11.707583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:11.723038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684e275-af21-49f9-972e-4a2a9b2d6be0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:11.774472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:11.775442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:11.775920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:11.776387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:11.777281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:11.778985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd93f3-2a57-402b-811e-df645ee12bae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:11.809719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 71/141] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:11 2022] POST /identity/v3/users => generated 283 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:11.938386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545299b-b78c-4372-b809-1d038743d55f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:11.939025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545299b-b78c-4372-b809-1d038743d55f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:11.939564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545299b-b78c-4372-b809-1d038743d55f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:11.941104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 71/142] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:11.958163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:11.958745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:11.959216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:11.983787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:11.985439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:12.071430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 72/143] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:12.357132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545299b-b78c-4372-b809-1d038743d55f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:12.489888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545299b-b78c-4372-b809-1d038743d55f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:12.521066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:12.534037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:12.534684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:12.534968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:12.535241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:12.552785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:12.557670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:12.558567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:12.559419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 72/144] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:12.566290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:12.674843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:12.704825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea123b0-ccae-46c4-8417-3baf93eb2583 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:12.752806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:12.753427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:12.753740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:12.754007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:12.758804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:12.759114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:12.760518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe85bc-7236-49a0-85b4-729a7d36bee1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:12.774244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 73/145] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:12.780301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:12.819473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095671e-ee92-4fe8-a1d4-4559ec640be6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:12.819782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095671e-ee92-4fe8-a1d4-4559ec640be6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:12.820014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095671e-ee92-4fe8-a1d4-4559ec640be6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:12.821507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 74/146] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:12.826749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:12.827040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:12.827325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:12.865138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:12.866390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:12.930309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ee0e8-df38-4b38-bd1e-bef29bdd72d6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:12.962476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 75/147] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:12.972072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:12.973671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:12.974164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:12.974451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:12.981739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:12.981739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad37767-9c43-4d33-bbf8-752805911d66 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:12.982726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 73/148] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:12 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:12.988118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:13.083453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:13.125669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e57d2-e427-4fb4-847e-2a785eb37add None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:13.174464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:13.175097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.175389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.175669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.238693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999b98-54b6-4e50-bd30-cb78a52f974b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.238965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999b98-54b6-4e50-bd30-cb78a52f974b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.239191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999b98-54b6-4e50-bd30-cb78a52f974b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.240795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 74/149] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:13.255819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:13.257308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:13.267894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.268179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.268480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.302125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:13.303525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:13.362464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:13.367370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:13.389778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 75/150] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:13.422811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 76/151] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:13.429167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:13.561469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ba9f8-87b1-4854-9c0b-b9e21a1bc8e9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:13.602235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:13.602884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.603170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.603450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.604593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:13.606105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:13.608071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 76/152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:13 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:13.613631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:13.676752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:13.770645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:13.800753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96840b-434f-4df5-99a3-e37caefe972c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:13.845396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ab25-4a20-474a-8c7e-f2e9336a5610 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:13.860720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:13.861764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.862267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.862807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.863963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:13.864568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:13.866253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:13.877937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 77/153] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:13 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:13.882405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:13.883069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:13.883357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:13.883705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:13.895813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:13.901220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:13.902134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:13.902876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 77/154] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:13.909446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:13.946677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.026332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa32dac-e051-4072-91c9-e1cb1c856497 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.074632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.075321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.075614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.075895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.076596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:14.076906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:14.078481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:14.082048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.111751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 78/155] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:14.143699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.282023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.290854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7adb-100b-4a96-ba1f-81a75323381c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.324168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1989181-b7d6-4fd8-9ec3-354575828678 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.344388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.352835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.353884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.354821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.355183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.356645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:14.358789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:14.361202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 79/156] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:14.367281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.409295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.409908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.410422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.411037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.428573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:14.433838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:14.434968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:14.435938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:14.439181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 78/157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:14.445972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.547391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe3759-ccd5-4a83-8e81-3e6e6c60b8fa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.595323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.596360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.597007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.597440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.621593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44dcaa-7b8c-4573-bfbd-2be1d6d376ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.634146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.668461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.669086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.669365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.669639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.670337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:14.670642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:14.672303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b966f8-7910-4724-b652-2be54f85cc3d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:14.683845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 79/158] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:14.686702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da37e-5fac-4110-8845-8b21c32b93fb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.686833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:14.687330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b818a-9bc2-4802-99e6-57946173daf5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:14.687433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da37e-5fac-4110-8845-8b21c32b93fb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.687961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da37e-5fac-4110-8845-8b21c32b93fb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.689512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 80/159] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:14.690849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223859-28bf-432a-92f6-54786b108504 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.691349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223859-28bf-432a-92f6-54786b108504 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.692393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223859-28bf-432a-92f6-54786b108504 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.697865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 81/160] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:14.698903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223859-28bf-432a-92f6-54786b108504 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.744392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.815218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 80/161] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:14.816514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.821273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.821273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.838059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:14.839337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:14.845044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223859-28bf-432a-92f6-54786b108504 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:14.900568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:14.902183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.902465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.904128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] PATH_INFO: `/v3/users/9ee5beb6969e4bab9f23c826894f841a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.906682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] RBAC: Authorizing `identity:get_user(user_id=9ee5beb6969e4bab9f23c826894f841a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:14.909131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a07d0-e2a9-4c60-b551-ccaecac957f1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:14.911650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 82/162] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/users/9ee5beb6969e4bab9f23c826894f841a => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:14.914717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:14.915000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:14.915280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:14.915707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:14.934670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 81/163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:14.936922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:14.955726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:14.958939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:15.087908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 83/164] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:15.088838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.121317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a30c7c-6ea6-4b88-b75e-864067562f93 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.147721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:15.162791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.163352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.163631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.163904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.165020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:15.166734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:15.168587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 82/165] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:14 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:15.174415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.256813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:15.298757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3747e90-3f03-4d69-b519-219db85274f7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.325865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b50f38-582a-4fde-9c52-cba66dfdcde2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.344633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.345269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.345548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.345863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.347124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:15.348831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:15.351344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 84/166] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:15 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:15.358350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.388313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.389150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.389537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.389901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.390725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:15.391189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:15.392831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:15.405733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 83/167] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:15 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:15.491722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:15.492994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.498942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29573-3bf0-45b8-83be-1313e7746fcf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.554664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.555734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.556218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.556668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/users/9ee5beb6969e4bab9f23c826894f841a/roles/b80f3217f46f47eba80ebbe5518c4bee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.564282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, user_id=9ee5beb6969e4bab9f23c826894f841a, role_id=b80f3217f46f47eba80ebbe5518c4bee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:15.566134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:15.591120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 85/168] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:15 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/users/9ee5beb6969e4bab9f23c826894f841a/roles/b80f3217f46f47eba80ebbe5518c4bee => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:15.676562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93004362-f066-4618-9f6e-a1c05a631b55 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.715513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:15.731067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.731731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.732018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.732324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.751910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:15.757635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:15.758563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:15.760022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 84/169] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:15.764800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.773001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:15.939330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb755f-288d-4be5-8a85-87e40f3f1824 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.981218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:15.981971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:15.982264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:15.982547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:15.983231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:15.983540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:15.984941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:15.987939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb6e72-9f4e-4d0a-909e-c18db9c0e891 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:15.997222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 86/170] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:16.009214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:16.050296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:16.060256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.060924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.061202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.061476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.074226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:16.080112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:16.080979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:16.082493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 85/171] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:16.087062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:16.188413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:16.217918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abc612-dbfa-4240-82da-afb6caa45e9a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:16.295471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28886d-3292-4998-bbbc-a1b0899be3e5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:16.308611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.309265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.309552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.309866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] PATH_INFO: `/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.314371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] RBAC: Authorizing `identity:get_user(user_id=a8e3d6ceca714fdb9b2c1a7b51c86f3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:16.315859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:16.317531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 87/172] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:16 2022] GET /identity/v3/users/a8e3d6ceca714fdb9b2c1a7b51c86f3c => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:16.323205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:16.371420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.371740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.372032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.372310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.373000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:16.373312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:16.374742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:16.392966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 86/173] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:16.398746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:16.522069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:16.546241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133205d-b75b-4910-83b4-cbd8de444a5b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:16.575649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07937e-2341-44ef-8404-b1dc98716d41 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:16.588121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:16.611647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.612273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.612559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.612837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.643385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.644026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.644312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.644585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] PATH_INFO: `/v3/users/fe943a904f9347cdbc2ea3c7493a9421` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.648436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] RBAC: Authorizing `identity:get_user(user_id=fe943a904f9347cdbc2ea3c7493a9421)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:16.649939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e185dd-eb7b-4bcc-a20f-8e1ba6cefce3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:16.651558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 87/174] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:16 2022] GET /identity/v3/users/fe943a904f9347cdbc2ea3c7493a9421 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:16.652702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078d29-5657-43ae-9519-a128ad931b8b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.652988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078d29-5657-43ae-9519-a128ad931b8b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.653264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078d29-5657-43ae-9519-a128ad931b8b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.662625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 88/175] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:16.663494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b078d29-5657-43ae-9519-a128ad931b8b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:16.747756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:16.749885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a448a-c06c-461d-a634-e03029d674eb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:16.851506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b078d29-5657-43ae-9519-a128ad931b8b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:16.965678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:16.966337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:16.966620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:16.966893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:16.996868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.046195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 88/176] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:17.047284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.047573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.047845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.106113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:17.107258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:17.108070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.141205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:17.142714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e3297-ed42-4135-a87d-01788612c607 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:17.196975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 89/177] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:17.197675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.283498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 89/178] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:17.285126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae5d30-6b3b-42cb-a974-5bf23dd4e079 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.285720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae5d30-6b3b-42cb-a974-5bf23dd4e079 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.286201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae5d30-6b3b-42cb-a974-5bf23dd4e079 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.292692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 90/179] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:17.294545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d8d37-29e7-495a-9ae5-ebe2746411a5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.295054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d8d37-29e7-495a-9ae5-ebe2746411a5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.295806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d8d37-29e7-495a-9ae5-ebe2746411a5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.297341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 91/180] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:17.298266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d8d37-29e7-495a-9ae5-ebe2746411a5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.393420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.408561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec39a5-19af-4eea-bf61-e43828cd6d15 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.426850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.433367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d8d37-29e7-495a-9ae5-ebe2746411a5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.446370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.447009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.447298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.447569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.448686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:17.450192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbda5c-26ba-46d9-8ed2-ef995b054ea5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:17.451831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 90/181] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:17.452920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.453192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.453469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.471072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:17.472254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:17.479748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.481041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.481532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.482032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] PATH_INFO: `/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.484033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8ed0e5be52c64ebd816a53beb4a8a9ce)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:17.486011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe993be1-61a5-44c2-b538-0acb610156d5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:17.489511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 92/182] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/projects/8ed0e5be52c64ebd816a53beb4a8a9ce => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:17.491298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.491767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.492239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.510736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:17.511919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:17.542113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 91/183] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:17.543017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.595655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 93/184] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:17.596905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.677297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.697216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5554dd0-d5c3-40c3-8bc9-6cde0e84d931 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.712068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36a36c-bd50-4f65-8a5b-82b433d06f04 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.724312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.751678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.752688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.753173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.753642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.765571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:17.771765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:17.772604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:17.773376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:17.775258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.775933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.776034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 94/185] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:17.776325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.776603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.777384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:17.777568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.777752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:17.779198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:17.797169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 92/186] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/role_assignments?user.id=a8e3d6ceca714fdb9b2c1a7b51c86f3c&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:17.798021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:17.912418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c27a1-e861-4a8b-b2ea-189ac8976b91 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.927615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b516-d170-47fa-9bce-30cd4761f4c5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:17.971747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:17.980494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.981875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.983619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.984834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:17.985865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=8ed0e5be52c64ebd816a53beb4a8a9ce, user.id=fe943a904f9347cdbc2ea3c7493a9421) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:17.986994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:17.988930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:17.995099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:17.996054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:17.997747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:17.998755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.000626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:18.000945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.002417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.004907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 95/187] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/role_assignments?user.id=fe943a904f9347cdbc2ea3c7493a9421&scope.project.id=8ed0e5be52c64ebd816a53beb4a8a9ce&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.006062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.013335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 93/188] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.014336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.092386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.172885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d7972-30d3-4339-9856-ca4a8835f2b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.177932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe5b86-7bdc-43ad-ae7f-bfc16a2312dd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.217217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.218027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.218473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.218885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.222179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.223109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.223556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.223981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.225292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.227179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.229535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 94/189] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.230342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.231955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:18.242341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:18.243268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:18.244140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 96/190] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:18.244821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.317462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.368671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c054bfb-4b00-4494-bbf3-f4fa3807521b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.371818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f7764-3ecb-4260-97f2-973afeb85c0f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.381454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.409488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.410238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.410501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.410823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.411106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.411727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.412014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.412293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.412945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.414453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03618946-097e-4fc3-b776-4e5a9db9329d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.426612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.428337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.430181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 95/191] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.431034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.440703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 97/192] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:18 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:18.440934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eac07-ca65-4c5c-970d-2af2857c07d9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.441223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eac07-ca65-4c5c-970d-2af2857c07d9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.441554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eac07-ca65-4c5c-970d-2af2857c07d9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.443347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 98/193] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:18.444305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eac07-ca65-4c5c-970d-2af2857c07d9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.555903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac7fe-fb0f-4985-94d6-bbfb376352b5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.559378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eac07-ca65-4c5c-970d-2af2857c07d9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.574754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.595930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.596562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.596866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.597128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.597886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:18.598201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.599619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32d1b9-b021-488d-b429-7dd2499b4a05 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.599844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.600539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.600879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.601167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.608791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 96/194] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.615492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.615818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.616100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.636735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:18.638025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:18.661901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.680715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.682192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.720042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 97/195] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:18.721296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.798758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 99/196] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:18.803905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.820865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.846635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e38d-c210-4aa9-ac60-56dd0fa683dc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:18.885312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:18.885994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:18.886281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:18.886561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:18.887703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:18.889161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:18.891244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 98/197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:18.896782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:18.927832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:18.951398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1c59-5e87-4865-9af4-1162d81a0653 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.009999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.010660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.010942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.011218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.013089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.014605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.016270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 100/198] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:19.022799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.032490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192549-b727-443b-96dc-e792c36b07ee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.069307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.070033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.070312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.070584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.133342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:19.151857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.153620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.158332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3cdd8-4f0a-410c-9a83-ec3390df40f0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.186881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:19.197331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.198529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.198953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.199395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.200353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=39ce7177340d407abd50f21f8c3b3de9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:19.200808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.202593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.209533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 101/199] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/role_assignments?user.id=39ce7177340d407abd50f21f8c3b3de9&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:19.210796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.274041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 99/200] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:19.283302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.368911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32f6c5-7a32-4e9f-969b-f0a3f4e99b35 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.440917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.443186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.443481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.443756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.458138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:19.464562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:19.465424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:19.466254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 102/201] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:19.475395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.479615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:19.485814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3729-132b-43d7-88ec-ad0372edb18d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.516630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:19.532318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.532971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.533254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.533530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.536362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.538092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.540350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 100/202] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:19.545977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.669641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b1ac-f127-41b2-918a-9990ce68b517 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.712313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.712855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.713147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.713424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.714191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=39ce7177340d407abd50f21f8c3b3de9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:19.714508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.715959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.724966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea0f5-4813-426e-9112-318d9844e05c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:19.729960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 101/203] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/role_assignments?user.id=39ce7177340d407abd50f21f8c3b3de9&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:19.778363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:19.778995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:19.779277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:19.779555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:19.780238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:19.780548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:19.782047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:19.796452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 103/204] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:19.802530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:19.892901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:19.952387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2df29-ac0f-4241-b2b7-207a1b4f067d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:20.002365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:20.002939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:20.003227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:20.007345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] PATH_INFO: `/v3/users/9ee5beb6969e4bab9f23c826894f841a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:20.007345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] RBAC: Authorizing `identity:get_user(user_id=9ee5beb6969e4bab9f23c826894f841a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:20.007345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:20.008489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 102/205] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:19 2022] GET /identity/v3/users/9ee5beb6969e4bab9f23c826894f841a => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:20.013968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:20.102486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:20.142226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a7c5c-d84e-4919-88af-30c3835efa85 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:20.196274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:20.196910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:20.197199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:20.197480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:20.315585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:20.317656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:20.418044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:20.459054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 104/206] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:20.465053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:20.609387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6c2a5-de04-4963-bc04-078c18af36aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:20.637866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:20.648523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:20.649363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:20.649650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:20.649962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:20.651166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:20.652650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1be75-a6a5-47d8-b12b-72308118eb55 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:20.654317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 103/207] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:20 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:20.662308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:20.811478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bca71-7431-4f64-a1ea-d0ec7c1a997f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:20.865971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:20.866570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:20.866851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:20.867124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:20.867914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, scope.project.id=7caa789bf4cc450d92cfac6640938018, user.id=9ee5beb6969e4bab9f23c826894f841a) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:20.868228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:20.869725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bb11d-9e8d-45ec-acf0-198456122d58 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:20.881950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 105/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:20 2022] GET /identity/v3/role_assignments?user.id=9ee5beb6969e4bab9f23c826894f841a&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:21.199504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc146981-1512-48ca-8f8d-a7072b81c763 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.199793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc146981-1512-48ca-8f8d-a7072b81c763 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.200095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc146981-1512-48ca-8f8d-a7072b81c763 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.201750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 104/209] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:21.207808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc08db-889e-45a8-b36c-9efd01dfd45d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.208267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc08db-889e-45a8-b36c-9efd01dfd45d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.208360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc08db-889e-45a8-b36c-9efd01dfd45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.226885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-67cc08db-889e-45a8-b36c-9efd01dfd45d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:21.228098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67cc08db-889e-45a8-b36c-9efd01dfd45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:21.252452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:21.306783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 106/210] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:21.411884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea7fd6-ff16-42e2-abaa-7506976b8dd8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.412568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea7fd6-ff16-42e2-abaa-7506976b8dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.413041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea7fd6-ff16-42e2-abaa-7506976b8dd8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.414908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 105/211] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:21.421240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.421808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.422308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.444484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:21.445990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:21.531952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 107/212] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:21.599221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea7fd6-ff16-42e2-abaa-7506976b8dd8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:21.702244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea7fd6-ff16-42e2-abaa-7506976b8dd8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:21.736925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:21.737820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.738291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.738774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.740034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:21.741626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:21.743432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 106/213] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:21.749678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:21.840166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:21.858083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:21.867986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2906-2248-48be-a4f5-b929dcedde12 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:21.921048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:21.925134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:21.927089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:21.927663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:21.928041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:21.929868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:21.931729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:21.974091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 108/214] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:21 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:21.982672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1da1-a442-476a-9cf9-2efdee43744f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.022894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:22.023847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.024276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.024694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.039001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ab2c8267-c1f5-420c-99ec-563427dd440a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:22.045497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:22.046684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:22.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:22.050326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 107/215] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:22.056266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.144336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fcf7-84e2-4967-a74b-d6d105eab632 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.144584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fcf7-84e2-4967-a74b-d6d105eab632 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.145381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70fcf7-84e2-4967-a74b-d6d105eab632 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.147018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 108/216] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:22.153378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45388-a276-4311-b028-46c5d911acb6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.153684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45388-a276-4311-b028-46c5d911acb6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.153977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45388-a276-4311-b028-46c5d911acb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.180781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e9c45388-a276-4311-b028-46c5d911acb6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:22.182472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9c45388-a276-4311-b028-46c5d911acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:22.189298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0c419-8e75-440b-8425-a789e00bfe08 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.233547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:22.234217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.234468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.234743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.235415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:22.235721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:22.237734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:22.253888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 109/217] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:22.259977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.269229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 109/218] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:22.275344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:22.312130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb947787-de74-43cd-a366-f11102b07ab2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.312416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb947787-de74-43cd-a366-f11102b07ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.312706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb947787-de74-43cd-a366-f11102b07ab2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.314253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 110/219] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:22.320084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.320747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.320747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.339761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:22.341082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:22.396586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b79ed6-7be9-43b5-9c87-d7808cee4690 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.408150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:22.432200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 111/220] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:22.438119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:22.438947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.439231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.439508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] PATH_INFO: `/v3/users/8bc9072348aa40aa946c86c7ddc122a6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.456661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=8bc9072348aa40aa946c86c7ddc122a6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:22.458230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:22.460324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 110/221] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/users/8bc9072348aa40aa946c86c7ddc122a6 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:22.466011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.571977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230a339-3106-4d96-a82e-5633cbea291c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.580653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.624175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:22.624832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.625071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.625346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.703749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:22.722277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:22.736134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:22.737762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:22.775485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394147-f6c4-450c-a641-e1a60632a0f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.822658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:22.823554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:22.823839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:22.824571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:22.838293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:22.844360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 112/222] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:22.844485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:22.845300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:22.846137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:22.846732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.850179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 111/223] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:22.851877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:22.954585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:22.974594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a8866-e076-4a3d-b1f8-73071d7dffb4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:22.980372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50796202-8d2a-43eb-a345-8c9e95141ffd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.005313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:23.015832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.016800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.017229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.017642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.019090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.021352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.023233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 112/224] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:23.024203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.030265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.031300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.031864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.032352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.044556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:23.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:23.051218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:23.052006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:23.059268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 113/225] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:23.060453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.145127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac81995-9c2e-4171-b444-537d5dd66275 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.185116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49339e43-0677-441d-8dea-28cdedd8819e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.186629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.187590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.188196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.188688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.189635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:23.190178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.191979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.202607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 113/226] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:23.203806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.230837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.231891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.232517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.233014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.233975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=8bc9072348aa40aa946c86c7ddc122a6) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:23.234480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.236206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.242444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 114/227] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/role_assignments?user.id=8bc9072348aa40aa946c86c7ddc122a6&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:23.243601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.267487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:23.323811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:23.391178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bea1a-3b01-48f1-98ba-0f58f71a7816 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.428389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef578-b1c0-4e1f-9729-16dd7c58bbc3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.433435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.434621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.435837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.436314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.437925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.443322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.445689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 115/228] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:23.463320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.484091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.484638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.484922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.485198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.485952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:23.486268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.487693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.497169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 114/229] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:23.502776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.581827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:23.610457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b31a84-7d95-4e21-b54b-cad8b333783c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.652078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.652729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.652983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.653257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.656096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.657562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.661982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 116/230] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:23.665861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.723487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb1198-b16b-468c-b514-19c45895a845 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.761814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:23.794731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.795365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.795646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.795919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.801924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.803399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.805023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 115/231] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:23.810657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:23.835536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad825-fb7a-4ecf-b3e0-50731b5bfbeb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:23.875112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:23.875770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:23.876050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:23.876320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/39ce7177340d407abd50f21f8c3b3de9/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:23.883815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, user_id=39ce7177340d407abd50f21f8c3b3de9, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:23.885483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:23.914497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 117/232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:23 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/39ce7177340d407abd50f21f8c3b3de9/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:23.982699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b3ca0-1296-4691-9151-f89e999a958a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:24.030566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:24.031244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.031532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.031875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.033132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:24.034624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701c62b-d9bc-4cf7-b356-d0e131066fb4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:24.036589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 116/233] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:23 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:24.042364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:24.045220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:24.184969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24d30d-b977-4ce9-bc93-937bffd21217 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:24.225719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:24.226399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.226689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.226968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/39ce7177340d407abd50f21f8c3b3de9/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.229165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, user_id=39ce7177340d407abd50f21f8c3b3de9, role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:24.230694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc3aa0-7ac8-4bcb-a76d-2efab3ea1f27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:24.250220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 118/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:24 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/39ce7177340d407abd50f21f8c3b3de9/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:24.670617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edb9c-6427-43f4-8505-558a0f73bdb9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.671140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edb9c-6427-43f4-8505-558a0f73bdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.671140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edb9c-6427-43f4-8505-558a0f73bdb9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.672673 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 117/235] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:24.679298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc897c-911b-4256-844d-de5c35a0b33d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.680243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc897c-911b-4256-844d-de5c35a0b33d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.680739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc897c-911b-4256-844d-de5c35a0b33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.700162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0abc897c-911b-4256-844d-de5c35a0b33d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:24.701571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0abc897c-911b-4256-844d-de5c35a0b33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:24.760006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:24.771302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 119/236] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:24.780865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9412-af6b-469e-925b-5d9c0aa3f767 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.781453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9412-af6b-469e-925b-5d9c0aa3f767 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.781963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9412-af6b-469e-925b-5d9c0aa3f767 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.783492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 118/237] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:24.789500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:24.790062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:24.790547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:24.808937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:24.810510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:24.921773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 120/238] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:25.050447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9412-af6b-469e-925b-5d9c0aa3f767 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.120406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:25.207862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9412-af6b-469e-925b-5d9c0aa3f767 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:25.253685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:25.254897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:25.255183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:25.255530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:25.273873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:25.279439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:25.280354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:25.281581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 119/239] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:25.286906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.304864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.474675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8faa-a543-467a-86b1-659bfa9c4dcd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:25.496096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736cbb-ad71-42b0-8534-a98a55756d85 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:25.518344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:25.523439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:25.547466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:25.548111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:25.548399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:25.548681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:25.549482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:25.549931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:25.551928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:25.565075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:25.565767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:25.566056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:25.566940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:25.568245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:25.571882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 121/240] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:25.572182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:25.574022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 120/241] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:25.578650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.583435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.728982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9642a-7ad4-4b5b-b981-e7753937803b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:25.768184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:25.768809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:25.769096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:25.769404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:25.774191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:25.775667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:25.800891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 121/242] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:25 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:25.807854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f31f-2486-415e-a34d-9cd35a5a1431 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:25.888039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:25.889066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:25.889547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:25.890351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] PATH_INFO: `/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:25.916360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] RBAC: Authorizing `identity:get_user(user_id=4ccc15fb47ac4deea884d288cc2bbcf1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:25.918234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c44eae-0f7a-42cf-a8c4-2873ee4bcaae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:25.920235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 122/243] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:25 2022] GET /identity/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1 => generated 297 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:25.926412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:25.986004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:26.014656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0832-764c-4cde-acad-42f4f8f619d8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.016575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0832-764c-4cde-acad-42f4f8f619d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.017063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0832-764c-4cde-acad-42f4f8f619d8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.020004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 123/244] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:26.029387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.029740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.029973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.050371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:26.052184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:26.097857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:26.109364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3851e-edec-42e3-8f93-5fb62e9d3fd5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:26.140092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 124/245] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:26.154225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:26.154842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.155128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.155404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.231423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:26.232975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:26.273356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:26.343945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 122/246] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:26.349959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:26.457970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:26.460842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e59b51-0dd7-4efa-be79-c450dd099641 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:26.503337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:26.504550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.504834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.505107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.508258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:26.508258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:26.509397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 125/247] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:26 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:26.515911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:26.568417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:26.592197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d494a-ba6d-457d-90fd-ed4c64161583 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:26.638891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:26.639800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.640089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.640368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.644201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab8cb0-cfa9-4673-907b-0412736cb99f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:26.649928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:26.656817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:26.664086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:26.664987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:26.666238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 123/248] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:26.672048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:26.692178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:26.693091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.693650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.693951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.694959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=4ccc15fb47ac4deea884d288cc2bbcf1) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:26.695294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:26.697732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5dfcd-29fb-48ab-9d27-9f1062b98414 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:26.707680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 126/249] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:26 2022] GET /identity/v3/role_assignments?user.id=4ccc15fb47ac4deea884d288cc2bbcf1&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:26.707680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22154-cbe4-433b-93f5-9d2c756f3bf1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.707916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22154-cbe4-433b-93f5-9d2c756f3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.707984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22154-cbe4-433b-93f5-9d2c756f3bf1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.709257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 127/250] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:26.715093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.715419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.715650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.735920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:26.737145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:26.821539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a69d11-8507-491d-be4d-9d5974b5b4e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:26.851547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 128/251] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:26.898614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:26.899267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:26.899594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:26.899879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:26.900578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:26.900889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:26.902371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:26.921452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 124/252] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:26.928426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:26.967378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:27.045265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d20c2f-d76b-4ff9-81cf-f4da14183d47 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:27.089248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:27.089839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.090123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.090394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] PATH_INFO: `/v3/users/8bc9072348aa40aa946c86c7ddc122a6` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.091534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] RBAC: Authorizing `identity:get_user(user_id=8bc9072348aa40aa946c86c7ddc122a6)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:27.092989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:27.094897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 129/253] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:26 2022] GET /identity/v3/users/8bc9072348aa40aa946c86c7ddc122a6 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:27.100895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:27.199187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:27.244521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295320c3-444e-4c87-b2b5-f710793dadf0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:27.285820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:27.286473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.286760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.287039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.288189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:27.289669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:27.291364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 125/254] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:27 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:27.297133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:27.349615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:27.417104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f61ed-139b-473f-8783-fc305a77defb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:27.430190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:27.452895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:27.453667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.453998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.454274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/8bc9072348aa40aa946c86c7ddc122a6/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.456182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, user_id=8bc9072348aa40aa946c86c7ddc122a6, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:27.458315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7f85d-ff4c-48b2-9907-b4daec61f370 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:27.480951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 130/255] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:27 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/8bc9072348aa40aa946c86c7ddc122a6/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:27.482463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b936419-774e-438b-9df1-3a9cfeb242a9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.484080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b936419-774e-438b-9df1-3a9cfeb242a9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.484367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b936419-774e-438b-9df1-3a9cfeb242a9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.486391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 131/256] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:27.493548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.493873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.494124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.532222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:27.533448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:27.570196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e214a-47a3-4bb1-adbf-fc15dbd6db6c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:27.615142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:27.624333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:27.624981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.625261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.625534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.638127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:27.643728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:27.644671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:27.645439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 126/257] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:27.650792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:27.680560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 132/258] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:27.807372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293aec7-f0b5-4b82-b832-7747c2a1af42 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:27.844075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:27.844688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:27.844952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:27.845228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:27.845951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:27.846258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:27.848353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:27.857444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 127/259] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:27.862838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:27.965961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:27.980875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91d94f-e2b2-4031-8a06-b2686aecb966 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:28.031292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:28.031945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:28.032290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:28.032572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:28.033766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:28.035253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:28.037208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 133/260] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:27 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:28.042446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.114371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:28.166954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393d093-16d4-4690-99e7-3e8b9defeaf7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:28.212271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:28.212886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:28.213172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:28.213448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:28.277117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.325696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:28.327568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:28.455743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:28.479865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ef8-7a3a-4599-aa5a-6d0dfb44cbf0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:28.539178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:28.539902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:28.540287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:28.540543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:28.556774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:28.570018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:28.570900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:28.571678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 134/261] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:28.577319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.579720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 128/262] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:28.585129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.609046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:28.736387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd4806-eac7-4085-b1a5-514a0afe0396 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:28.782761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930bc8f-85ac-47b9-80fc-e5e7344ee20e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:28.793261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:28.794177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:28.794654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:28.795114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:28.795984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:28.796493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:28.802119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:28.818159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 135/263] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:28.823804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.850399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:28.851064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:28.851362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:28.851637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:28.852809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:28.854319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:28.856308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 129/264] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:28.867025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:28.907014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:28.960641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:28.984329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511edc42-38be-4ef9-8fc1-0c38e7c5cc65 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:29.051472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:29.052210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.052584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.052974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] PATH_INFO: `/v3/users/39ce7177340d407abd50f21f8c3b3de9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.054236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] RBAC: Authorizing `identity:get_user(user_id=39ce7177340d407abd50f21f8c3b3de9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:29.055859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:29.057859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 136/265] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/users/39ce7177340d407abd50f21f8c3b3de9 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:29.065087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:29.066426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c188-6b60-4493-9ef1-b4a3896145eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:29.133074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:29.134038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.134539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.134949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.135826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=39ce7177340d407abd50f21f8c3b3de9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:29.136276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:29.137988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:29.150189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 130/266] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:28 2022] GET /identity/v3/role_assignments?user.id=39ce7177340d407abd50f21f8c3b3de9&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:29.200491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6643-53ec-4e7c-a697-99e78b174938 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:29.243620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:29.244263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.244542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.244813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.270116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:29.335195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:29.336634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:29.490391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 137/267] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:29.496253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:29.629490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:29.657848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c9216-6c73-462d-9242-103cdcf1c908 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:29.711643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:29.712315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.712603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.712882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.714067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:29.715545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aad613-5fe8-4342-b4dd-4d6c5f71dbdf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:29.717505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 131/268] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:29 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:29.724302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:29.818372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:29.819582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cb-b50f-4da3-b8db-a3fde85eaa07 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.819879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cb-b50f-4da3-b8db-a3fde85eaa07 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.820160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cb-b50f-4da3-b8db-a3fde85eaa07 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.821927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 132/269] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:29.831486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b62bc2-e0d6-420a-89ee-b4483035b713 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.832383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b62bc2-e0d6-420a-89ee-b4483035b713 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.832551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b62bc2-e0d6-420a-89ee-b4483035b713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.862444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-90b62bc2-e0d6-420a-89ee-b4483035b713 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:29.863695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-90b62bc2-e0d6-420a-89ee-b4483035b713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:29.878090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee720755-d18b-4c26-93f6-20ccfaa1ccf1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:29.921366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:29.922364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.922821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.923352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.924233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=39ce7177340d407abd50f21f8c3b3de9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:29.924691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:29.926381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a6405-4778-4a81-ad59-028df10ad9e8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:29.934005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 138/270] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:29 2022] GET /identity/v3/role_assignments?user.id=39ce7177340d407abd50f21f8c3b3de9&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:29.935454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd8eb-bd0f-4c0b-8526-10fa4f30c097 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.935748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd8eb-bd0f-4c0b-8526-10fa4f30c097 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.936030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd8eb-bd0f-4c0b-8526-10fa4f30c097 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.937239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 139/271] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:29.941807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97a39-c3d9-4600-97e8-04386beb1918 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.941807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97a39-c3d9-4600-97e8-04386beb1918 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.941807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97a39-c3d9-4600-97e8-04386beb1918 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.941807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 140/272] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:29.945812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.946298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.946750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.962889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 133/273] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:29.964446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:29.964884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:29.965299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:29.967463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:29.968751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:29.989984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:29.991829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:30.072912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 141/274] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:30.115665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 134/275] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:30.310565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.380746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.416420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.432659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.444345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7279f8-4f95-40a7-87e8-0b59a0e1c2b7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.482593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:30.483233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.483542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.483818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.494201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:30.499064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:30.499908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:30.500682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 142/276] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:30.501311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.525045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848ee7e-3c1e-4aab-924a-be48c0784a40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.558732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:30.559368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.559654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.559930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.561094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:30.562527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518fa7-2c31-4592-872d-03917d878eac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:30.564102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 135/277] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:30.564102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ac7ea-cbd5-464e-ad70-0f2faa5ec810 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.564102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ac7ea-cbd5-464e-ad70-0f2faa5ec810 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.564102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ac7ea-cbd5-464e-ad70-0f2faa5ec810 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.570193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 136/278] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:30.570193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ac7ea-cbd5-464e-ad70-0f2faa5ec810 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.602610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aaa9f-2368-414f-80a8-ed1b92b057f9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.604949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.637058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:30.637690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.638007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.638284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.648116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:30.648820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:30.653271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:30.654192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:30.655065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 143/279] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:30.655696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.674962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ac7ea-cbd5-464e-ad70-0f2faa5ec810 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.709929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:30.710533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.710810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.711082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.711767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:30.712075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:30.713461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27383e94-e66e-4b7e-ac59-1668d3681d08 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:30.721235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 137/280] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:30.722370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.722658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.722984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.741037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:30.742251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:30.754326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2eb4f0-1fca-4777-b223-3ca572b3d845 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.787534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:30.788129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:30.788402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:30.788678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:30.789314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:30.790781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:30.809462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 138/281] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:30.810451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.820996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 144/282] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:30 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:30.821768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:30.857479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.899180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:30.948753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3de19-c745-4f23-ae93-775bb32e54ca None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:30.969491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c9772-dc64-44e1-ac03-d2daaa66b187 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.000906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.002104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.003010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.003791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.004908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:31.005834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.007908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.010533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.010956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.011238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.011508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] PATH_INFO: `/v3/users/55d5eda623ca4b27ace2c1df588276ec` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.018905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 139/283] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.024358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.035524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] RBAC: Authorizing `identity:get_user(user_id=55d5eda623ca4b27ace2c1df588276ec)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.037011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.040399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 145/284] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:30 2022] GET /identity/v3/users/55d5eda623ca4b27ace2c1df588276ec => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.045215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.140555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:31.160331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84d149-04ae-455d-8d9f-d3c9355f91fa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.167457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd0f2b-ad6a-4da1-879f-3bbb5f6fae27 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.195523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:31.202692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51844887-70db-495d-85d0-050da25a0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.203466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51844887-70db-495d-85d0-050da25a0800 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.203767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51844887-70db-495d-85d0-050da25a0800 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.204008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51844887-70db-495d-85d0-050da25a0800 None admin] PATH_INFO: `/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.205284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51844887-70db-495d-85d0-050da25a0800 None admin] RBAC: Authorizing `identity:get_user(user_id=4ccc15fb47ac4deea884d288cc2bbcf1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.207304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51844887-70db-495d-85d0-050da25a0800 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.208779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.209246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 140/285] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.210142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.210427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51844887-70db-495d-85d0-050da25a0800 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.210960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.211471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.212342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, user.id=55d5eda623ca4b27ace2c1df588276ec, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:31.212789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.215973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.224865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 146/286] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/role_assignments?user.id=55d5eda623ca4b27ace2c1df588276ec&scope.system=all&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:31.226622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.341904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51844887-70db-495d-85d0-050da25a0800 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.397197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.397798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.398089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.398368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.400412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810e132-3342-4487-b407-a47ac6db7465 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.426429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:31.434812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:31.435762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:31.436526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:31.443237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 141/287] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:31.448001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.470551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.471204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.471491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.471772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.472999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.474534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.476254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 147/288] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.483466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.510066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:31.555851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:31.633038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5f3b0-f8cf-4090-86e6-24025137b70e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.664385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd454c37-5eee-439c-a240-6600980d6554 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.695802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.696930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.698612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.699084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.700735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:31.703916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.705543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.720355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 142/289] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.726567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:31.733067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.736421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.736968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.737447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/4ccc15fb47ac4deea884d288cc2bbcf1/roles/b80f3217f46f47eba80ebbe5518c4bee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.741665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, user_id=4ccc15fb47ac4deea884d288cc2bbcf1, role_id=b80f3217f46f47eba80ebbe5518c4bee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.743324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.764250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 148/290] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:23:31 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/users/4ccc15fb47ac4deea884d288cc2bbcf1/roles/b80f3217f46f47eba80ebbe5518c4bee => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:23:31.879845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8505c-d0f0-447f-9cde-d3507ccab40f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:31.917928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:31.933215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:31.934311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:31.934789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:31.935319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] PATH_INFO: `/v3/users/8bc9072348aa40aa946c86c7ddc122a6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:31.936735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] RBAC: Authorizing `identity:get_user(user_id=8bc9072348aa40aa946c86c7ddc122a6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:31.938452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:31.945404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 143/291] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/users/8bc9072348aa40aa946c86c7ddc122a6 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:31.946786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:32.028809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:32.067005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3515-1885-4a22-af79-ca4ab1437eef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:32.102847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:32.103466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:32.103750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:32.104027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:32.173582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:32.175021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:32.274090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:32.305727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 149/292] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:32.315165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:32.458325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe3093-291c-4477-942b-604d89dbe072 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:32.498973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:32.499626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:32.499912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:32.500187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:32.501341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:32.502823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14626071-9c18-40f4-8c21-4acb40baef3d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:32.504494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 144/293] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:32 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:32.510493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:32.611425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b61fe1-bf8b-43fb-91f4-69c197077dcc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:32.653054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:32.653667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:32.653987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:32.654267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:32.656791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=8bc9072348aa40aa946c86c7ddc122a6) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:32.656791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:32.656791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3628f-db8b-4f00-a95b-234d2e3dbb2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:32.663078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 150/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:32 2022] GET /identity/v3/role_assignments?user.id=8bc9072348aa40aa946c86c7ddc122a6&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:33.423634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fbe8-2819-4e8b-ba6e-63ca78b2dd33 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:33.424194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fbe8-2819-4e8b-ba6e-63ca78b2dd33 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:33.424194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fbe8-2819-4e8b-ba6e-63ca78b2dd33 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:33.426459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 145/295] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:33.432283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d1212-1691-4d92-a7d5-034d7e03fb5a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:33.432893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d1212-1691-4d92-a7d5-034d7e03fb5a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:33.432893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d1212-1691-4d92-a7d5-034d7e03fb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:33.452287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-076d1212-1691-4d92-a7d5-034d7e03fb5a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:33.453512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-076d1212-1691-4d92-a7d5-034d7e03fb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:33.481483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:33.527338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 151/296] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:33.543085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74bc1-38b6-4e3e-9739-b5554fbfc280 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:33.543563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74bc1-38b6-4e3e-9739-b5554fbfc280 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:33.543970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74bc1-38b6-4e3e-9739-b5554fbfc280 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:33.545624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 146/297] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:33.551491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:33.551857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:33.552204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:33.571529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:33.572742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:33.640141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 152/298] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:33.804449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74bc1-38b6-4e3e-9739-b5554fbfc280 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:33.867493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:33.912908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:33.917633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74bc1-38b6-4e3e-9739-b5554fbfc280 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:33.958230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:33.959225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:33.959722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:33.960183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:33.971034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:33.977522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:33.978856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:33.979949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:33.987254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 147/299] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:33.988992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.030902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.035585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f287a1-0f74-4c41-9c82-753d3ded34f1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.072465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.073533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.074045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.074511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.075872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:34.077499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:34.084501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 153/300] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:34.085924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.120859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877012b-cc76-4d7e-8f5d-8db15bbe3972 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.141302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.161305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.161958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.162251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.162533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.163279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:34.163538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:34.165571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8c5c0-411d-428d-a382-34e4da54527f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:34.180667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 148/301] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:34.182424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00624-1993-4001-b959-18e3373a0994 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.182721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00624-1993-4001-b959-18e3373a0994 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.183000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00624-1993-4001-b959-18e3373a0994 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.184546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 149/302] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:34.185238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af00624-1993-4001-b959-18e3373a0994 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.203225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cebbbc-9740-4e23-b58b-8e6dc0a25b8e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.239286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.239942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.240224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.240497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] PATH_INFO: `/v3/groups` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.241114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:34.242570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00e0ca-8f29-403d-8bd1-9f7804506ca5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:34.261195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 154/303] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Sep 12 15:23:34 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:23:34.264035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca634-49a1-4b51-8c03-54545e79551a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.264586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca634-49a1-4b51-8c03-54545e79551a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.265032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca634-49a1-4b51-8c03-54545e79551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.297410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3e4ca634-49a1-4b51-8c03-54545e79551a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:34.298865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3e4ca634-49a1-4b51-8c03-54545e79551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:34.304724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af00624-1993-4001-b959-18e3373a0994 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.320203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.345056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.345746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.346025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.346305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] PATH_INFO: `/v3/users/7ce7face8cd343919eea4d6b9d06ab72` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.364281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce7face8cd343919eea4d6b9d06ab72)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:34.365840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80cbf5-cf3e-41d9-9a9a-af0e36b32ce6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:34.369150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 150/304] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/users/7ce7face8cd343919eea4d6b9d06ab72 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:34.369150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859012e-cad3-4e9e-af42-4ec07205fb4a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.369150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859012e-cad3-4e9e-af42-4ec07205fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.369400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859012e-cad3-4e9e-af42-4ec07205fb4a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.374747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 151/305] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:34.374747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859012e-cad3-4e9e-af42-4ec07205fb4a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.410471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 155/306] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:34.411603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.411963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.412339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.431319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.431859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:34.432744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:34.510634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 156/307] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:34.515051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859012e-cad3-4e9e-af42-4ec07205fb4a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.557795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.557795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.557795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.557795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.558181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, user.id=7ce7face8cd343919eea4d6b9d06ab72, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:34.558181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:34.558467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:34.563994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 152/308] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/role_assignments?user.id=7ce7face8cd343919eea4d6b9d06ab72&scope.system=all&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:34.688826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.775088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.794435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dca99a-500f-47dd-b43e-10750d4cb138 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.803989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.835240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.836409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.837299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.837760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.853843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:34.860810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:34.861932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:34.863586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:34.865860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 157/309] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:34.872946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.923046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308d00-71ce-4ae2-be69-c1e50468297b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:34.924969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:34.964450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:34.965323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:34.965766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:34.966176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:34.977692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:34.983273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:34.984550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:34.985498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:34.988000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 153/310] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:34.993122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:34.998443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52011ed-a16e-4ce8-93ab-e4c32aa9f280 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.045267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.046188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.046612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.047026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.047840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:35.048286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.050419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.060407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 158/311] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:35.066048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.070472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:35.132390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec4d967-6467-4178-9e45-dcb9aa281bbf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.178622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c442-b96d-4f1e-8a49-27a29a1dbd4f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.190013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.191671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.192130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.192575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.194457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:35.194945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.196982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.209207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 154/312] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:35.217822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.218904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.219547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.219895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.220175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] PATH_INFO: `/v3/users/55d5eda623ca4b27ace2c1df588276ec` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.221356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] RBAC: Authorizing `identity:get_user(user_id=55d5eda623ca4b27ace2c1df588276ec)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.223080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.225030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 159/313] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:35 2022] GET /identity/v3/users/55d5eda623ca4b27ace2c1df588276ec => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:35.230214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.243874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:35.307216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:35.326876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c2a5c-23a1-41b0-85ae-97ff21a4789a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.340835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3415-4bfc-43c0-a10b-f05d90cdddfd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.363393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.364365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.364839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.365301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] PATH_INFO: `/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.366688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] RBAC: Authorizing `identity:get_user(user_id=4ccc15fb47ac4deea884d288cc2bbcf1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.368322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.374794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 155/314] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:35 2022] GET /identity/v3/users/4ccc15fb47ac4deea884d288cc2bbcf1 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:35.375732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.388957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.389927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.390401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.390823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] PATH_INFO: `/v3/system/users/55d5eda623ca4b27ace2c1df588276ec/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.392436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=55d5eda623ca4b27ace2c1df588276ec, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.394100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.405783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 160/315] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Sep 12 15:23:35 2022] PUT /identity/v3/system/users/55d5eda623ca4b27ace2c1df588276ec/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:35.482854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:35.493608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02db09-f99c-4980-986a-4a8bba9353de None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.534479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.535147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.535430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.535705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.613401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.614994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.711959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:35.722626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 156/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:35.728551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.833849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a32b-a754-4c8b-a719-5ba0e7bce2bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:35.869354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:35.870244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:35.870694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:35.871112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:35.872378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:35.874001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918a8a-12d2-4c1f-92cc-fdcb503c484e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:35.876068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 161/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:35 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:35.883077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:35.991954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13466ba0-35be-4fcf-986c-462d77b0cb9e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:36.027326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:36.027947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:36.028233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:36.028512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:36.029244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24, user.id=4ccc15fb47ac4deea884d288cc2bbcf1) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:36.029557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:36.031046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bbe68-a223-4d5b-9c08-57be95f946a3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:36.038064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 157/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:23:35 2022] GET /identity/v3/role_assignments?user.id=4ccc15fb47ac4deea884d288cc2bbcf1&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:36.568913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71229-4bed-49fa-927d-bd9d32d2b2d5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:36.569550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71229-4bed-49fa-927d-bd9d32d2b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:36.569760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71229-4bed-49fa-927d-bd9d32d2b2d5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:36.571253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 162/319] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:36.576157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f17ea-ce34-456a-a5d5-35dc19308830 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:36.576605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f17ea-ce34-456a-a5d5-35dc19308830 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:36.576790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f17ea-ce34-456a-a5d5-35dc19308830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:36.594562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a12f17ea-ce34-456a-a5d5-35dc19308830 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:36.595739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a12f17ea-ce34-456a-a5d5-35dc19308830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:36.646480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:36.664581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 158/320] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:36.824477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922413-f75d-4173-a076-d32b3cce8511 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:36.824986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922413-f75d-4173-a076-d32b3cce8511 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:36.825058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922413-f75d-4173-a076-d32b3cce8511 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:36.826537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 163/321] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:36.831406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:36.831997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:36.832406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:36.850758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:36.851943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:36.919722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 159/322] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:36.931413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c922413-f75d-4173-a076-d32b3cce8511 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:36.935151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:37.034341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c922413-f75d-4173-a076-d32b3cce8511 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.067726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.068353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.068687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.068966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.070334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:37.071956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:37.077309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 164/323] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:37.079067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:37.177994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d698a-8212-4c53-a594-d1043d918ee8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.194979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:37.195143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:37.215038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.215606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.215890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.216159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] PATH_INFO: `/v3/groups` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.216791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:37.218305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:37.241905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 160/324] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Sep 12 15:23:37 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:37.275334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:37.311607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5cb6-dd4c-4a20-8748-78781ef16394 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.348916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.349641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.349958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.350238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.370772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:37.376812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:37.377776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:37.382609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 165/325] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:37.384513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:37.493573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d74a5-5e82-49bd-944c-0a045d22eb72 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.531871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.532624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.532914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.533196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.533901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:37.534214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:37.536254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f5ef1-be8c-4790-a8ef-4d4f05599440 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:37.545610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 161/326] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:37.551378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:37.672108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55453d-99f6-4d02-a2cb-c06a0cea3fa3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.692234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a1be-3276-43bd-8a75-d720b830ebb1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.692760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a1be-3276-43bd-8a75-d720b830ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.692760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a1be-3276-43bd-8a75-d720b830ebb1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.694576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 162/327] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:37.700712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.701000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.701280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.703696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:37.710272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.711116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.711405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.711683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] PATH_INFO: `/v3/users/7ce7face8cd343919eea4d6b9d06ab72` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.713819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce7face8cd343919eea4d6b9d06ab72)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:37.715288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:37.721606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 166/328] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:37 2022] GET /identity/v3/users/7ce7face8cd343919eea4d6b9d06ab72 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:37.725279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:37.728022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:37.729249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:37.772172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:37.817208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 163/329] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:37.839797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657bd0a-4345-4f7a-bfbd-8125950cbc13 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:37.875685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:37.876650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:37.877083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:37.877500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] PATH_INFO: `/v3/system/users/7ce7face8cd343919eea4d6b9d06ab72/roles/b80f3217f46f47eba80ebbe5518c4bee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:37.879731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7ce7face8cd343919eea4d6b9d06ab72, role_id=b80f3217f46f47eba80ebbe5518c4bee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:37.881307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:37.891799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 167/330] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Sep 12 15:23:37 2022] PUT /identity/v3/system/users/7ce7face8cd343919eea4d6b9d06ab72/roles/b80f3217f46f47eba80ebbe5518c4bee => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:23:38.133176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:38.238760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5736338-3c99-4d60-bdb6-4e2d7791db60 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:38.275735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:38.276378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:38.276662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:38.276934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:38.287836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:38.292986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:38.294004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:38.294779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:38.301090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:38.387762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:38.438277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a133-0dc0-4ba1-97c0-d207ce93c8d9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:38.483394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:38.484016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:38.484347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:38.484625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:38.485355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:38.485663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:38.487149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:38.501820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 168/332] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:38.504129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:38.536097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:38.621664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fda1-64d1-4792-be12-1caf71b821b9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:38.659484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:38.660601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:38.660887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:38.661165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] PATH_INFO: `/v3/users/55d5eda623ca4b27ace2c1df588276ec` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:38.662352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] RBAC: Authorizing `identity:get_user(user_id=55d5eda623ca4b27ace2c1df588276ec)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:38.663814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528a354-48d9-494b-b785-fa56f7a6fbcb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:38.665728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:38 2022] GET /identity/v3/users/55d5eda623ca4b27ace2c1df588276ec => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:38.671324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:38.793221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f31a8-c25e-45c8-b9bc-81ffbe9db2bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:38.826109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:38.834758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:38.835594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:38.835875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:38.836145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:38.836873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] RBAC: Adding query filter params (role.id=86d42fd805a14d5d8e83fbb0653bdb3e, user.id=55d5eda623ca4b27ace2c1df588276ec, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:38.837177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:38.838797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5109c3-6482-4d93-9b84-2c4392829a5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:38.856921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 169/334] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Sep 12 15:23:38 2022] GET /identity/v3/role_assignments?user.id=55d5eda623ca4b27ace2c1df588276ec&scope.system=all&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:40.065010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61192-fd50-437f-82a0-330c3bf5f824 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.065520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61192-fd50-437f-82a0-330c3bf5f824 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.065624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61192-fd50-437f-82a0-330c3bf5f824 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.067114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 166/335] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:40.077121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.077121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.077121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.094388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:40.095917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:40.143584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eedff-a654-4be0-9f6c-da112bd65bf4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.144240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eedff-a654-4be0-9f6c-da112bd65bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.144721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eedff-a654-4be0-9f6c-da112bd65bf4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.146435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 167/336] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:40.151793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.152322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.152800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.183079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 170/337] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:40.184370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:40.185885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:40.291512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 168/338] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:40.554075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:40.668093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc64408-c8e0-4a64-9c83-3836ffc0af0d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:40.685053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:40.713757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:40.714747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.715231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.715705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.728166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:40.733536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:40.734427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:40.735192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:40.737412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 171/339] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:40.744062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:40.766984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:40.790139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:40.905443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87c57d-3ad9-4735-af8e-56329ec381fe None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:40.930516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30f0b-52dd-4751-a6bf-8f756ed9b7fb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:40.983964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:40.984186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:40.984610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.984900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.985181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.985863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:40.986019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:40.986240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:40.986710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:40.987325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:40.987862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:41.003641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:41.008401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 169/340] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:41.009824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:41.010702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:41.011492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:41.018311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 172/341] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:41.021177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:41.025263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:41.095711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:41.194533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90319a-b815-4dc2-88e0-c1645d2fbf37 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:41.219929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:41.236108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:41.236724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.237001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.237270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.238003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:41.238305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:41.239678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977868dd-fe71-4bbc-94b0-f08be0e7ef38 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:41.241595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fced2f-862d-4b35-a3bd-d81eb13b8517 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:41.248361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 173/342] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:41.249697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc71e-4614-4436-8aff-9bac90a00873 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.254195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc71e-4614-4436-8aff-9bac90a00873 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.254628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cc71e-4614-4436-8aff-9bac90a00873 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.256049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 174/343] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:41.261902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.262261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.262461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.294480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:41.295693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:41.311433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:41.312045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.312327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.312599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.433102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 175/344] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:41.439205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:41.460205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:41.461610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315f3f-1959-4049-ba9e-034a4dcc3e4c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:41.486121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:41.599303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce446953-a32b-44d2-bd27-f1261ec305f9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:41.629625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:41.646408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:41.647031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.647310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.647585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.654678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 170/345] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:41.662268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2df-4c7b-4010-9a79-53908be9af22 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.662575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2df-4c7b-4010-9a79-53908be9af22 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.662853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad2df-4c7b-4010-9a79-53908be9af22 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.664202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 171/346] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:41.664934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad2df-4c7b-4010-9a79-53908be9af22 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:41.784067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:41.786282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:41.813117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad2df-4c7b-4010-9a79-53908be9af22 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:41.830653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:41.871628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:41.872271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.872551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.872825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.874090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:41.875534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba5faa-4c92-4416-aa52-758922e0c45b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:41.879988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 172/347] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:41 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:41.879988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:41.880968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:41.881845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:41.914449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:41.917475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:41.943832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 176/348] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:41.945249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:41.992769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.016441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 173/349] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:42.017649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.088296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea933-46c6-4b97-9921-c0815160913b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.123837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.132831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f163e21-8784-4a4a-8fd3-9a0bf6a0d4b9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.147981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.148663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.148950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.149228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.165608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:42.171257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:42.172134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:42.173046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 177/350] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:42.178379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.178662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.179748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.180222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.180713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.188032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.190049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.192109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 174/351] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.193224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.251421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.302498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c00638-95c4-486b-bb93-d4192caa91cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.313646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553fc33-88b8-44a0-b583-3342be560bca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.346262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.347522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.348234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.348618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.350640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.353205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.355925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 178/352] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.357057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.357357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.358164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.358618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.359027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.359831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:42.360295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.361840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.374575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 175/353] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.375414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.440719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.493514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb445-4bf9-4d9e-9a4c-c59a085e8171 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.495106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672da73-b0e1-4aff-b7ef-1370d1aa8b3b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.511867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.540442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.541115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.541402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.541676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.542024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.542628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.542909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.543181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.544073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=7caa789bf4cc450d92cfac6640938018) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:42.544668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.548804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.553194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 179/354] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.554779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.559728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:42.566322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:42.567289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 15:23:42.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:42.571309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 176/355] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:42.571960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.711171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b1fde-1c7b-4b17-98ea-47d68bb50010 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.729644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471107-6eec-4e03-89de-795831e782d8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.733144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.756676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.757807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.758098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.758372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.759843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.761310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591ce83-b738-4d82-8a50-f7aa3ca69f5d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.764724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 180/356] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.765929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e22ec-0e98-4108-9021-5099643502ce None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.766214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e22ec-0e98-4108-9021-5099643502ce None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.766488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e22ec-0e98-4108-9021-5099643502ce None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.767949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 181/357] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:42.768653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e22ec-0e98-4108-9021-5099643502ce None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:42.784755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.785773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.786635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.786916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.848131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:42.882312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.883792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b81cf-acd5-4714-96a8-ceef819814c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.911385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e22ec-0e98-4108-9021-5099643502ce None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:42.953350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:42.954142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:42.954430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:42.955010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:42.956219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:42.956356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:42.960742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:42.968054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 182/358] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:42.968851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.011118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 177/359] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:43.012522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6333d37-1431-4987-be8a-f4498733a019 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.012817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6333d37-1431-4987-be8a-f4498733a019 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.013106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6333d37-1431-4987-be8a-f4498733a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.033143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.035825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c6333d37-1431-4987-be8a-f4498733a019 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:43.037270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c6333d37-1431-4987-be8a-f4498733a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:43.102704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98310bef-6333-4df4-b4cb-f67c8c14b46c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.132274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 178/360] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:43.134610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da62e-79b2-4584-9619-ac185df6e56f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.135086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da62e-79b2-4584-9619-ac185df6e56f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.135735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da62e-79b2-4584-9619-ac185df6e56f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.137812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 179/361] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:43.138925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057da62e-79b2-4584-9619-ac185df6e56f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.141313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.142068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.142428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.142787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.143072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.143646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=7caa789bf4cc450d92cfac6640938018) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:43.144161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:43.145679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:43.158256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 183/362] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:42 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:43.159226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.327976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057da62e-79b2-4584-9619-ac185df6e56f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.340561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66008a-b0b4-4be5-8528-a4920d69e3f2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.358716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.392808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.393454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.394331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.394618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.395772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:43.397277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1469ca-e2b6-4f3d-8d68-7be76030b141 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:43.398952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 184/363] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:43.400169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.400456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.400730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.419657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:43.420855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:43.424896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.425589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.425900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.426174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] PATH_INFO: `/v3/users/7ce7face8cd343919eea4d6b9d06ab72` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.428586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] RBAC: Authorizing `identity:get_user(user_id=7ce7face8cd343919eea4d6b9d06ab72)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:43.430235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:43.432974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 180/364] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/users/7ce7face8cd343919eea4d6b9d06ab72 => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:43.433908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.531860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 185/365] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:43.532709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.567611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.606436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d1d5-a899-4de3-9a3e-062c8a05cf63 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.647183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.647824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.648103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.648375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.649479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:43.651443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:43.653517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 181/366] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:43.654516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.734808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755263-d26e-457d-a15f-c46b7e67b189 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.750605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.771679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c88a-6563-48a3-9fd0-7f458a6da4f4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:43.789624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.790275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.790561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.790834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.791560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] RBAC: Adding query filter params (role.id=b80f3217f46f47eba80ebbe5518c4bee, user.id=7ce7face8cd343919eea4d6b9d06ab72, scope.system=all) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:43.791871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:43.793328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:43.810795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 186/367] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/role_assignments?user.id=7ce7face8cd343919eea4d6b9d06ab72&scope.system=all&role.id=b80f3217f46f47eba80ebbe5518c4bee => generated 510 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:43.811245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:43.812114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:43.812583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:43.813101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:43.814158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.826521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:43.832004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:43.832957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:43.833815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:43.836772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 182/368] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:43.841750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:43.845221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:43.989165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531efb3-f0eb-4b34-84e6-ce1175314802 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.029828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.031238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.031704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.032150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.033062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:44.033564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.036241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.042871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 187/369] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:44.043953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.064112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcc43c-6b2f-4faa-974f-c3aa6b3c28f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.083868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.162663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.163107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.163397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.163677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.164374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:44.164694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.166157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.170484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dd37e-3d99-4539-b653-dc1bd386ca6d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.188261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 183/370] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:44.194395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.212455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.213409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.213889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.214494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.227555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:44.234558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:44.235491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:44.236272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:44.239386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 188/371] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:44.245519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.317859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.355883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.378991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09a9e8-933e-4ba6-b015-ce2fedaba8d4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.393347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b787-ca58-4a27-941f-3545bdca37fc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.424040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.424040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.424040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.425220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.426415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:44.426728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.428552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.438497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.439553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.439904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 189/372] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:44.440308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.441174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.452332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.524733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.526326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.586878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b85aee-dbf4-41b2-aafe-4122d87f05ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.588473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.626673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.627549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.627975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.628941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.637797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 184/373] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:44.642311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.660585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.754833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.756292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.832683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64159f-cbb5-4639-ae12-6e7926c92cff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:44.895926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 190/374] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:44.898876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.903768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:44.904674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:44.905154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:44.905639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:44.914486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:44.916301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:44.918284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 185/375] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:44.923827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:44.965947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:44.995186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:45.035199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefca7-e9fa-4619-b175-4634626f30a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:45.098485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:45.099032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.099317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.099590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] PATH_INFO: `/v3/projects/45e559d5550544c28dd39b8ed456ebde` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.100781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] RBAC: Authorizing `identity:get_project(project_id=45e559d5550544c28dd39b8ed456ebde)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:45.102236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:45.104151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 191/376] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/projects/45e559d5550544c28dd39b8ed456ebde => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:45.104262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4f978-d888-4a11-8e64-2cdcc69d5e9b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:45.109553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:45.206493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:45.207163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.207451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.207731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.214582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:45.216071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:45.217938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 186/377] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:44 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:45.221758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c5fba-74a7-437b-b8b3-24e9dc55ed8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:45.223974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:45.254672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:45.261339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:45.262053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.262331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.262734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] PATH_INFO: `/v3/groups/418193fbc6e649f89fa4ebb168822e8e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.270698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] RBAC: Authorizing `identity:get_group(group_id=418193fbc6e649f89fa4ebb168822e8e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:45.272192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:45.274352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 192/378] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:45 2022] GET /identity/v3/groups/418193fbc6e649f89fa4ebb168822e8e => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:45.280874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:45.394198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754bfa-b7cb-4b87-a3f1-76c8f2c3c4d1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:45.430058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:45.435042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:45.435588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e7a7-9b0f-43ad-861e-9d71b8c2fd03 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:45.435896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.436125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.436401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.437143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] RBAC: Adding query filter params (group.id=418193fbc6e649f89fa4ebb168822e8e, role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=45e559d5550544c28dd39b8ed456ebde) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:45.437453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:45.439554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b92fd8-5ec5-4e0d-aefb-0d3092ca9132 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:45.446564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 193/379] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:45 2022] GET /identity/v3/role_assignments?group.id=418193fbc6e649f89fa4ebb168822e8e&scope.project.id=45e559d5550544c28dd39b8ed456ebde&role.id=c939df765d074277be1dc85b4ec66f05 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:45.451418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033084a-6c9a-4913-b99b-19ea9dd52d80 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.451734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033084a-6c9a-4913-b99b-19ea9dd52d80 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.452076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033084a-6c9a-4913-b99b-19ea9dd52d80 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.453890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 194/380] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:45.459862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.460177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.460412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.482658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:45.483882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:45.494540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:45.495183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:45.495472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:45.495751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:45.496503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:45.496818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:45.498319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:45.504681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 187/381] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:45 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:45.552216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:45.564122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 195/382] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:45.879797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.002211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde8d97-87ed-4ded-9d54-d83e0c0b2d8c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.043495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.044153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.044448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.044725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.058980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e4864c99-d820-4911-9dc9-c9bbcad39126 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:46.064392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:46.065240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:46.066486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 188/383] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:46.071779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.221526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3574b3-ae02-43a5-a166-4ef6b9856b02 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.221937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3574b3-ae02-43a5-a166-4ef6b9856b02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.222752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3574b3-ae02-43a5-a166-4ef6b9856b02 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.224359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 189/384] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:46.230336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.230626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.230902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.235889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff99e2-dafa-4553-a731-5c89216a1a40 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.250149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:46.251364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:46.268178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:46.299323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.300207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.300324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.300537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.301216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:46.301530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:46.303046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:46.321581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 196/385] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:46.327101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.330737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 190/386] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:46.385458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:46.452064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640bfaa-853c-4ff9-b753-8f1f30f4448f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.489651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.490300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.490586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.490860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.492035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:46.493498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:46.503296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 197/387] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:46 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:46.503456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.637892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e695-0438-470b-a35e-75dcb7de2f1d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.679675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.680644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.681125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.681593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.682968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:46.684773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:46.686936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 191/388] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:46 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:46.692527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.816233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:46.840991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:46.848953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8ef4e-d571-44d6-8627-101116df2d52 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.869355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:46.935471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.936172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.936459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.936736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.939601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3ab7-6db9-4214-8088-bc120016e31f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:46.942999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, group_id=72a81ba6c39f4bea87e6268a8d68886b, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:46.944731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff307c-76eb-4034-9598-68af946fd133 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:46.967584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 198/389] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Sep 12 15:23:46 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:46.968914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4db81-7020-4f00-a2c1-7cd18b108fba None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.969211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4db81-7020-4f00-a2c1-7cd18b108fba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.969485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4db81-7020-4f00-a2c1-7cd18b108fba None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.975449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 199/390] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:46.978660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.979213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.979546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:46.996580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:46.997405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:46.997849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:46.998127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.002041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:47.003319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:47.021588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:47.028204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:47.029105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:47.029947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:47.030540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 192/391] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:47.037665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.122402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 200/392] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:47.165890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:47.189475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b0c38-d5e8-4cc4-ac5a-99a1df317c8e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:47.227314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:47.228004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:47.228333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:47.228608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.229331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:47.229642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:47.231714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:47.243463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 193/393] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:47.250959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.383192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:47.388501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9aa38-3001-4b8b-882d-c98d5578806d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:47.427218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:47.427857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:47.428139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:47.428420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.429501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:47.431048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:47.432969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 201/394] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:47 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:47.439003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.570074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5914-b524-41f1-865b-bf15b11704f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:47.614999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:47.615643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:47.615926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:47.616199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.617319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:47.618799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:47.620706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 194/395] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:47 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:47.626082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.643050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.663893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:47.789503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a204f6-4a70-4204-8579-404175fe3956 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:47.829196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ad99-d906-406b-9115-ad400eae8cea None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:47.844348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:47.845027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:47.845312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:47.845593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.849880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] RBAC: Authorizing `identity:create_grant(project_id=7caa789bf4cc450d92cfac6640938018, group_id=72a81ba6c39f4bea87e6268a8d68886b, role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:47.851356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:47.873827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 202/396] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Sep 12 15:23:47 2022] PUT /identity/v3/projects/7caa789bf4cc450d92cfac6640938018/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:47.887536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:47.888251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:47.888595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:47.888878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:47.901879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:47.907590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:47.908447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:47.909233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 195/397] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:47.916395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:47.948254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:48.032330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d70ca-5382-40ad-985f-f294c1472722 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:48.086504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:48.087144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.087431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.087711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.088400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:48.088717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:48.090146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:48.099735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 203/398] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:48.105550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:48.176838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:48.228929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c028b90-1327-424a-b1bd-3700cf12d8c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:48.270415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:48.271062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.271354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.271631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.272968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:48.274491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d075e1-ad37-43f3-91ee-5418c11ef620 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:48.276484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 196/399] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:48 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:48.281893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:48.398150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:48.419914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1dac8-3c14-463e-9d12-8cc234de4494 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:48.457786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bcbf5-5c7f-4581-9cbb-d5ba0f51765c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.458062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bcbf5-5c7f-4581-9cbb-d5ba0f51765c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.458272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bcbf5-5c7f-4581-9cbb-d5ba0f51765c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.459609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 197/400] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:48.465888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b98-c2d0-41aa-a941-9dafc7d1346b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.466171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b98-c2d0-41aa-a941-9dafc7d1346b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.466447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b98-c2d0-41aa-a941-9dafc7d1346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.478005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:48.478712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.479001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.479278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.481631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:48.483128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:48.488570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 204/401] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:48 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:48.492812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ebe79b98-c2d0-41aa-a941-9dafc7d1346b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:48.494538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:48.497970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ebe79b98-c2d0-41aa-a941-9dafc7d1346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:48.604204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 198/402] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:48.606438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cef358-41a2-4809-a7fa-c539c12c46f1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.607517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cef358-41a2-4809-a7fa-c539c12c46f1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.608012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cef358-41a2-4809-a7fa-c539c12c46f1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.614291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 199/403] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:48.616387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.617130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.618099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.638397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88489942-5a5a-4fcc-a7b4-42634048d806 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:48.642825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:48.644770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:48.709258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:48.709926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:48.710210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:48.710485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:48.714335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, group_id=72a81ba6c39f4bea87e6268a8d68886b, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:48.715856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:48.741872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 205/404] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Sep 12 15:23:48 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:23:48.747183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:48.764090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 200/405] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:48.914360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:48.938068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:49.031840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309ed49-9dc2-4ca9-9fa3-84a4abeec39f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.068018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.068709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.068992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.069568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.079783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:49.085307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:49.086198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:49.087132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 206/406] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:49.094383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.096133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.232431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:49.269209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be33d-7b99-459c-9ace-5229a2571365 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.272640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402a1d-ef36-4dbf-b483-e503d3900a22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.342068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.342691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.342914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.343176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.346368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.347064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.347351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.347626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.348304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:49.348615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:49.350021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:49.361015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:49.366797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:49.367896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:49.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:49.370989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 207/407] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:49.378631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.384443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 201/408] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:49.391035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.517216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:49.526794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01286c97-be51-4ae2-97e1-52d01e25481d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.593689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72525ea-31be-418b-8e7c-46f9ca95f851 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.606419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.607050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.607337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.607621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.608478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:49.608790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:49.610222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:49.616791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:49.622836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 208/409] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:49.630655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.651728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.652447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.652645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.652922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] PATH_INFO: `/v3/groups/418193fbc6e649f89fa4ebb168822e8e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.655417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] RBAC: Authorizing `identity:get_group(group_id=418193fbc6e649f89fa4ebb168822e8e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:49.655524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:49.657491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 202/410] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/groups/418193fbc6e649f89fa4ebb168822e8e => generated 229 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:49.663383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.791680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7f178-0861-412f-8c97-d0975eace0e4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.851419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:49.855203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47985113-30b0-467d-9921-c3035668ab7b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:49.873342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.874076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.874361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.874638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] PATH_INFO: `/v3/projects/45e559d5550544c28dd39b8ed456ebde` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.875795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=45e559d5550544c28dd39b8ed456ebde)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:49.877261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:49.879482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 203/411] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/projects/45e559d5550544c28dd39b8ed456ebde => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:49.892170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:49.917696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:49.918364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:49.918644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:49.918916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:49.922490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:49.922490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:49.923584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 209/412] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:49.929793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:50.006119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:50.045613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba80aec-32fc-4779-b244-220ef5f17644 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:50.049411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f87d11-dae6-41eb-b6b6-b8b489c3b4a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:50.087468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:50.088551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.089012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.089584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.091184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:50.092868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:50.093293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd90220-3aa2-4581-afae-37c5663bf1c7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:50.093684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.094065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.094409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] PATH_INFO: `/v3/projects/45e559d5550544c28dd39b8ed456ebde/groups/418193fbc6e649f89fa4ebb168822e8e/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.096220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 210/413] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:49 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:50.096345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] RBAC: Authorizing `identity:create_grant(project_id=45e559d5550544c28dd39b8ed456ebde, group_id=418193fbc6e649f89fa4ebb168822e8e, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:50.098382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39622fc4-0e0f-4867-9f6d-31489e9dbc0f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.098830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39622fc4-0e0f-4867-9f6d-31489e9dbc0f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.099279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39622fc4-0e0f-4867-9f6d-31489e9dbc0f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.100618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 211/414] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:50.102750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39622fc4-0e0f-4867-9f6d-31489e9dbc0f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:50.103416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f04c11-109e-4367-97f8-d543a06ddf9b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:50.118666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 204/415] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Sep 12 15:23:49 2022] PUT /identity/v3/projects/45e559d5550544c28dd39b8ed456ebde/groups/418193fbc6e649f89fa4ebb168822e8e/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:50.127046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20335f-3d30-4f0e-a10d-2500f8c430d7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.127327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20335f-3d30-4f0e-a10d-2500f8c430d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.127607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20335f-3d30-4f0e-a10d-2500f8c430d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.161651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-af20335f-3d30-4f0e-a10d-2500f8c430d7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:50.162899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-af20335f-3d30-4f0e-a10d-2500f8c430d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:50.221209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39622fc4-0e0f-4867-9f6d-31489e9dbc0f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:50.259706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:50.262843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:50.263808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.264337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.264787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.267182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24, group_id=72a81ba6c39f4bea87e6268a8d68886b, role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:50.269491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:50.273317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:50.288061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 212/416] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Sep 12 15:23:50 2022] PUT /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24/groups/72a81ba6c39f4bea87e6268a8d68886b/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:23:50.304309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 205/417] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:50.617172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:50.619452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d247-682b-4666-8531-79654631ca12 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.619741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d247-682b-4666-8531-79654631ca12 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.620022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d247-682b-4666-8531-79654631ca12 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.621690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 206/418] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:50.626962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.627252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.627537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.660693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:50.661728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:50.735048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90a1d6-5bc5-4e20-8809-5058edcca115 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:50.775387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:50.776403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.776883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.777359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.780207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 207/419] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:50.789173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:50.794446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:50.795314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:50.796097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:50.798668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 213/420] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:50.805746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:50.959605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3aa44-dd1a-48cb-a7ce-78387f5d1149 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:50.974318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:50.995561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:50.996237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:50.996528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:50.996816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:50.997476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:50.997812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:50.999254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:51.008924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 208/421] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:51.014583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.092827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.110924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.151775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded44d-de26-485e-91aa-1fda840ce161 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:51.195299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:51.196027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.196378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.196743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.260698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4af77-64b0-4937-9df2-4ca9407b44b0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:51.273511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:51.275385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:51.305503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:51.306525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.306888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.307230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.327304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8e3adada-496b-46ed-94c5-febce87c10e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:51.335179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:51.336083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:51.336895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:51.341653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 209/422] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:51.343395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94cd25-6429-4850-a67c-4e86ff54d90d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.343686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94cd25-6429-4850-a67c-4e86ff54d90d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.343967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94cd25-6429-4850-a67c-4e86ff54d90d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.350165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.350394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 210/423] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:51.352367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.354110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.354391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.374743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:51.375946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:51.413136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.424270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 214/424] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:51.426057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.461814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 211/425] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:51.463135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.538877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd24e-95ff-4eac-8347-c9abbad6643b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:51.575592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:51.576202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.578492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.578492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.578492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:51.578492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:51.579402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:51.593228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 215/426] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:51.594505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.615710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.664471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fd0e-0b85-4416-9285-00f5a74edc3c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:51.702105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcb27e-7d5c-4b60-8557-4a441da6a0c2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:51.737874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:51.738490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.738870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.739157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.742368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:51.743010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:51.743289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:51.743561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:51.744706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:51.746186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:51.747797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 212/427] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:51.753333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.820715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:51.822223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:51.843342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.846296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:51.940240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 216/428] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:51.941466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:51.962237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25f327-0dec-4a1a-8ede-a37de593ef53 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.009058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.010351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.011261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.012006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.013976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.016401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.019239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 213/429] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:52.020222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.069983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dbaa-b3fa-4980-a7c4-ad9623615776 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.090630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.111918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.113610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.114124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.114599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.127127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:52.129439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943407c-9fc5-47f8-8811-37ffaac83739 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.130978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:52.134542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:52.135402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 15:23:52.136181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:52.138475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 217/430] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:52.139491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.167394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.168214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.168606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.168890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] PATH_INFO: `/v3/projects/7caa789bf4cc450d92cfac6640938018` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.170164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] RBAC: Authorizing `identity:get_project(project_id=7caa789bf4cc450d92cfac6640938018)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.171747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.178619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 214/431] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/projects/7caa789bf4cc450d92cfac6640938018 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:52.179455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.258531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51043f-73bc-4888-b494-801e4bea378f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.288662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42d21b-a371-4e00-843e-a1295e40ab71 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.302378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.303443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.303935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.304536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.305554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=7caa789bf4cc450d92cfac6640938018) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:52.306121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.308473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.316195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 218/432] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:52.317996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.330947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.332499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.333017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.333552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.335282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:52.335778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.337868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.351743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 215/433] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:52.359345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.395416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.438938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.518357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6798c65-e3b5-4a0c-aad7-f809239cf89d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.560786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.561422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.561722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.562006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.562298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560505-481f-49f1-921e-7b109ef536ef None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.563121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.564800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.571512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 219/434] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:52.573564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.605484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.606244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.606532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.606809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.705407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.707421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9f3ad-6938-4143-b81a-d76bf9e5ee40 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:52.707800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.724398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.749464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:52.753517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:52.754331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.754761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.755179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.757115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=7caa789bf4cc450d92cfac6640938018) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:52.757592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:52.759200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f53460-3172-4bf6-9e2d-00bb1355806c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:52.769905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 220/435] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=7caa789bf4cc450d92cfac6640938018&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:52.815178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d929ff-c586-4dc7-b7f2-24d160104648 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.816229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d929ff-c586-4dc7-b7f2-24d160104648 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.816503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d929ff-c586-4dc7-b7f2-24d160104648 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.818252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 221/436] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:52.825044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:52.825329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:52.825604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:52.849416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:52.850702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:52.862171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 216/437] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:52.868765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:52.959696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 222/438] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:23:53.000631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25161cdc-f491-4994-911a-91a6ac15b34b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.033252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.038318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.039437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.039962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.040635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.042158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.043619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d4abe-9744-4ee7-966a-68e9080b0b90 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.045515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 217/439] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:52 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:53.051255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.062866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def3c5c-4617-488e-8cce-871208b0a1ee None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.063436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def3c5c-4617-488e-8cce-871208b0a1ee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.063704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def3c5c-4617-488e-8cce-871208b0a1ee None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.065163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 218/440] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:23:53.070769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.071055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.071333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.098070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:53.099695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:53.194684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7242db-c2b7-4ec8-a4e1-0d7e8a06d259 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.196956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.198394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 219/441] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:53.235314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.235955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.236239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.236520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.237720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.239229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.241167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 223/442] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:53.247176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.248938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.381011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11363900-44e7-4270-a38e-a3e373a14b9c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.397023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443addf9-20c2-4a9d-9bfb-3df3c17ee513 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.422253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.422253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.422530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.422804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.433278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.436766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:53.442283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:53.443179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:53.443974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 224/443] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:53.445537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.446372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.446790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.447227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.447982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=86d42fd805a14d5d8e83fbb0653bdb3e, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:53.448301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.452727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.452852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.457043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 220/444] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=86d42fd805a14d5d8e83fbb0653bdb3e => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:53.481639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.491935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.571450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b6e6c-08a1-48e8-aefa-cef8184ea8fa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.597585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe3660-3d27-43a3-b4d5-77e192414a71 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.611594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.612262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.612674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.612956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.613643 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:53.613990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.615425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.624512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 225/445] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:53.631856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.639177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.639878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.640164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.640436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.651531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:53.656904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:53.657864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:53.658923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 221/446] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:53.663988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.733044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.744047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.752832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c5b83-2f5c-4e0e-b083-65da2d1cdf97 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.790317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128677-80c7-496e-b8a8-fbd2bb77a6dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.791539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.792271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.792549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.792934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.826335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.826953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.827233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.827519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:53.828189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:53.828500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.829942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.839003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 222/447] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:53.844833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.874588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:53.876041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:53.946616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36791bae-3b9f-41bb-beb7-ad40584632a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:53.962207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.966587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:53.974669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 226/448] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:23:53.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:53.980665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:53.981260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:53.981624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:53.981938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.048572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.050593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.076832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aef3b-90bc-4634-b178-8033ec34f4bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.110292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.110914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.111197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.111473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] PATH_INFO: `/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.113162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] RBAC: Authorizing `identity:get_project(project_id=a0aa1fbdab6a44e396cf24dd1ca45f24)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.114632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.116251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 227/449] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/projects/a0aa1fbdab6a44e396cf24dd1ca45f24 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:54.121641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:54.150163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 223/450] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:23:54.155660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:54.173367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:54.180145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:54.219037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd96014-10b6-43e6-81ff-38ec00d5bdaf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.252079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.253320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.253612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.253921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] PATH_INFO: `/v3/groups/72a81ba6c39f4bea87e6268a8d68886b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.255063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] RBAC: Authorizing `identity:get_group(group_id=72a81ba6c39f4bea87e6268a8d68886b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.256515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.258106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 228/451] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:54 2022] GET /identity/v3/groups/72a81ba6c39f4bea87e6268a8d68886b => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:54.259359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4b37-a983-424a-b897-b4d6e06d21ee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.263878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:54.293337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.293993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.294278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.294557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] PATH_INFO: `/v3/projects/45e559d5550544c28dd39b8ed456ebde` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.295676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] RBAC: Authorizing `identity:get_project(project_id=45e559d5550544c28dd39b8ed456ebde)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.297152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.298872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 224/452] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:23:54 2022] GET /identity/v3/projects/45e559d5550544c28dd39b8ed456ebde => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:54.305093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:54.363719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f235f0-0cef-4360-8e39-0b37bd3fb588 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.389234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:54.389510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:54.395720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.396315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.396600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.396870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.397586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] RBAC: Adding query filter params (group.id=72a81ba6c39f4bea87e6268a8d68886b, role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:54.397920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.399367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.404971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 229/453] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:54 2022] GET /identity/v3/role_assignments?group.id=72a81ba6c39f4bea87e6268a8d68886b&scope.project.id=a0aa1fbdab6a44e396cf24dd1ca45f24&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:54.405667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ac9c3-edd8-4358-b4a1-944050dd20a0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.443178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.443835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.444387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.444665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] PATH_INFO: `/v3/groups/418193fbc6e649f89fa4ebb168822e8e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.445890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] RBAC: Authorizing `identity:get_group(group_id=418193fbc6e649f89fa4ebb168822e8e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.447358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47540-782b-49a5-b709-ce5384b90dc9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.448976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 225/454] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Sep 12 15:23:54 2022] GET /identity/v3/groups/418193fbc6e649f89fa4ebb168822e8e => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:54.454893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:54.565678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71b4a-eefc-4ff9-84ba-13c259e817fa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:54.603000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194762be-844a-4a49-907c-341e9afb800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:54.603615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194762be-844a-4a49-907c-341e9afb800c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:54.603902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194762be-844a-4a49-907c-341e9afb800c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:54.604179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194762be-844a-4a49-907c-341e9afb800c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:54.604921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194762be-844a-4a49-907c-341e9afb800c None admin] RBAC: Adding query filter params (group.id=418193fbc6e649f89fa4ebb168822e8e, role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=45e559d5550544c28dd39b8ed456ebde) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:54.605218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194762be-844a-4a49-907c-341e9afb800c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:54.606697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194762be-844a-4a49-907c-341e9afb800c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:54.612513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 230/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Sep 12 15:23:54 2022] GET /identity/v3/role_assignments?group.id=418193fbc6e649f89fa4ebb168822e8e&scope.project.id=45e559d5550544c28dd39b8ed456ebde&role.id=c939df765d074277be1dc85b4ec66f05 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:57.277341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd3c3-ad47-4a08-a59d-f1f1980636fb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.278391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd3c3-ad47-4a08-a59d-f1f1980636fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.278898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd3c3-ad47-4a08-a59d-f1f1980636fb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.280435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 226/456] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:57.286430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104876-5718-47c8-9046-b96137088282 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.286829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104876-5718-47c8-9046-b96137088282 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.286982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104876-5718-47c8-9046-b96137088282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.314675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-08104876-5718-47c8-9046-b96137088282 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:57.316165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-08104876-5718-47c8-9046-b96137088282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:57.397623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:57.414044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 231/457] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:57.642041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cc080-239f-40e3-84e4-09a01dd402db None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.642706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cc080-239f-40e3-84e4-09a01dd402db None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.643195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cc080-239f-40e3-84e4-09a01dd402db None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.645459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 227/458] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:57.651059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.651598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.652098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.683057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:57.684586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:57.731170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03e90a-cc28-4010-adba-6ee7e3092d44 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.731784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03e90a-cc28-4010-adba-6ee7e3092d44 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.732260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03e90a-cc28-4010-adba-6ee7e3092d44 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.734096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 228/459] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:57.739582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:57.740040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:57.740585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:57.773265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:57.774459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:57.777589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 232/460] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:57.856281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:57.870963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 229/461] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:57.970003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe1ad4-2767-46e6-8c5c-d274b80c59c8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:57.998034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.015799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.016504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.016803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.017077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.028131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.029692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:23:58.046723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:58.047685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:23:58.048508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:23:58.048508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.048508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:58.051172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 233/462] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:58.054889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.127336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.154072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.187291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6473a8-7844-4696-adfa-eff28889a683 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.229553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.230276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.230558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.230827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.231470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:58.231770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.233157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e73c80-bc79-4ff2-90f5-862a5347a094 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 230/463] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f281-1070-4655-8d80-a8409418e4e3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f281-1070-4655-8d80-a8409418e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f281-1070-4655-8d80-a8409418e4e3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 231/464] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:58.248909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f281-1070-4655-8d80-a8409418e4e3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.267451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22395de-7f4d-4a52-861f-0b48aba953e8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.305676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.306339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.306817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.307110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.320450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.323065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:23:58.340724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:58.341627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:23:58.342788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:23:58.342788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.342788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:58.347136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 234/465] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:58.347932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.355693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f281-1070-4655-8d80-a8409418e4e3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.382078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.398425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.399088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.399377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.399655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.410600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.412064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.414926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d99311c4-c5f3-4638-ac8b-03f37a3558fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:23:58.429660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:58.430580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:23:58.431484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:23:58.431484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:58.431484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:58.431484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 232/466] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:58.432055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.432491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.432766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.451812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:58.452994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:58.469552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd3739-d810-44b9-97ea-c4f2d6a7f38a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.513673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.514528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.515019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.515671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.516946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.518991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.523387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 233/467] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:58.524221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.550797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 235/468] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:58 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:58.551836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.661307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfba34-3069-4583-9fa1-0558218cbf72 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.664703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.697465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:58.708112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78a96-5187-44e6-a102-56e0b9f71fd0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.710218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.711094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.711516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.711979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.712832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:58.713279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.714865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fbf96-dac6-4a7e-970c-d96865b6303f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.728949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 234/469] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:58.735250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0856f-f2ab-4f54-a4d3-f0b5ed6d6534 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.735879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0856f-f2ab-4f54-a4d3-f0b5ed6d6534 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.736300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0856f-f2ab-4f54-a4d3-f0b5ed6d6534 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.739678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 235/470] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:23:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:23:58.740719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0856f-f2ab-4f54-a4d3-f0b5ed6d6534 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.750134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.750764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.751050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.751326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.752270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:58.752270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.753781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875705-ae6b-46bb-ad2c-66e47f4b825b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.767070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 236/471] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:23:58.768195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.768528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.768814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.787987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:23:58.791880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:23:58.873924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0856f-f2ab-4f54-a4d3-f0b5ed6d6534 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:58.877427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 237/472] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:23:58.878189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.913182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:58.913837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:58.914129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:58.914405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:58.915073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:58.916533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:58.941021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 236/473] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:58 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:58.942227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:58.986209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:59.002184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:59.062780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ab550-f774-4c5e-b2eb-ccc6099ce665 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.103789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.104900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.105381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.105934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.106809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.108532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.140411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d86f-57ac-4453-a747-07f6f983d33f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.144725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 238/474] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:58 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:59.191945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.193057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.193565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.194376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.209507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.211644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.229252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:23:59.230625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:59.231497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:23:59.232339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:23:59.232339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:23:59.232339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:23:59.238260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 237/475] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:59.244701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:59.352224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d3ca-561a-44ba-9934-e66d3cbbda57 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.374645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:59.383867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:59.388039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb5917-4524-4e29-8fc3-e8db45df960f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.396395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.397265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.397726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.398159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.399033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.400779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.410818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 15:23:59.410818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 15:23:59.410818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [parameters: {'id': '08e30e7c51b149bdb40d4b8ac6e2f7ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 15:23:59.410818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Sep 12 15:23:59.428535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return ref.to_dict() Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:23:59.429630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 15:23:59.430575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 15:23:59.431404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:23:59.432347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:23:59.433177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'id': '08e30e7c51b149bdb40d4b8ac6e2f7ce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:59.437520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 15:23:59.439653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.441622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 239/476] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Sep 12 15:23:59 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:23:59.441622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:59.441990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.441990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.441990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.441990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.442538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:59.442538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.443177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.459619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 238/477] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:23:59.466077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:59.583916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471122c9-a83d-4724-afa1-297cfee541bd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.587434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eeb36-4d24-4b9f-ab68-c4fb779c82c1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.627547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.628174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.628457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.628733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.632238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.633094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.633565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.634234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.635360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.637572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b7792-128c-41d2-893e-917866a0a056 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.640080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:23:59.645743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:23:59.646620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:23:59.647399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 240/478] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:23:59.653068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:23:59.667986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 239/479] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:23:59 2022] POST /identity/v3/users => generated 249 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:23:59.716673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:23:59.776724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2598c-3def-4ac7-8315-fba15622ce54 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:23:59.814216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:23:59.815210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:23:59.815712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:23:59.816155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:23:59.817000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:23:59.817489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:23:59.819342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66f458-8322-4c6e-ae50-f65f7d064e2e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:23:59.829286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 241/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:01.107421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9baef-41d1-48cf-ac1c-e47f80ef60c6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:01.108447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9baef-41d1-48cf-ac1c-e47f80ef60c6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:01.108941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9baef-41d1-48cf-ac1c-e47f80ef60c6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:01.110815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 240/481] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:01.116338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272772-1ec6-4df6-8f44-909fddf66b66 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:01.117115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272772-1ec6-4df6-8f44-909fddf66b66 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:01.117581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272772-1ec6-4df6-8f44-909fddf66b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:01.144234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b8272772-1ec6-4df6-8f44-909fddf66b66 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:01.145856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b8272772-1ec6-4df6-8f44-909fddf66b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:01.240251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 242/482] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:24:01.694452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9baef-41d1-48cf-ac1c-e47f80ef60c6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:01.707861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:01.804981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecb8e7-0a14-4d0f-963c-0e760749e848 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:01.805788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecb8e7-0a14-4d0f-963c-0e760749e848 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:01.806278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecb8e7-0a14-4d0f-963c-0e760749e848 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:01.808289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 243/483] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:01.812916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8366e7-b335-49eb-9479-afba8fcc46f5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:01.813208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8366e7-b335-49eb-9479-afba8fcc46f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:01.813487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8366e7-b335-49eb-9479-afba8fcc46f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:01.834417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9baef-41d1-48cf-ac1c-e47f80ef60c6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:01.844517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bd8366e7-b335-49eb-9479-afba8fcc46f5 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:01.845743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bd8366e7-b335-49eb-9479-afba8fcc46f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:01.902133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:01.902974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:01.903642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:01.903928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:01.904203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:01.929630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:01.940426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:01.941365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:01.942183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:01.946072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 241/484] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:01.947988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:01.975041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 244/485] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:02.030402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.068223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b336a5-9bf5-4b2b-b6a0-62295ee70986 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.071500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90951-d1b5-4f07-a035-0d6eea8718a9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.071797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90951-d1b5-4f07-a035-0d6eea8718a9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.072074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90951-d1b5-4f07-a035-0d6eea8718a9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.073540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 245/486] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:02.078909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449401fc-9941-4203-9487-1532283816f0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.079196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449401fc-9941-4203-9487-1532283816f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.079484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449401fc-9941-4203-9487-1532283816f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.104293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.104906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.105188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.105463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.106168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:02.106476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.107927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.111629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-449401fc-9941-4203-9487-1532283816f0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:02.113393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-449401fc-9941-4203-9487-1532283816f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:02.117291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 242/487] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:02.122768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.230244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f8b7-a803-43c4-84fb-da268e85cf85 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.260852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.267714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.268373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.268656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.268937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.274868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 246/488] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:24:02.294765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.296559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:02.315018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:02.316509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:02.317336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:02.317336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:02.317336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:02.323679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 243/489] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:02.325134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449401fc-9941-4203-9487-1532283816f0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.368123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.505053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449401fc-9941-4203-9487-1532283816f0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.510085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.555309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9345-5c6a-4325-9874-e9e870c8ba0a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.567005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.567872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.568325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.568767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.569885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:02.570382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.571913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.585221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 247/490] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:02.586586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.598929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.600061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.600349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.600626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.616781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:02.623760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:02.625202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:02.626063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 244/491] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:02.626711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.687035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.709825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afdad6-b61d-47c1-a6bf-83fd9076b38f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.744142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.744768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.745049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.745325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.747216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fab2b9-10d6-40cc-a5c5-2f645d0ff42b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.755251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:02.760497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:02.761491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:02.762305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 248/492] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:02.762936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.783354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.783678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.785009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.785290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.785562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.793046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.794519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f64bce75-c998-4934-bd1e-fec0bbf56ee0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:02.807025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:02.807916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:02.808801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 245/493] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:02.809471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1e0b-0a2f-4f0f-a5b1-17560357607d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.809689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1e0b-0a2f-4f0f-a5b1-17560357607d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.810655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1e0b-0a2f-4f0f-a5b1-17560357607d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.811923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 246/494] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:02.815363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c0f7-687e-4fbb-891b-64e3ef47fb4b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.815854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c0f7-687e-4fbb-891b-64e3ef47fb4b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.816131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c0f7-687e-4fbb-891b-64e3ef47fb4b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.817346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 247/495] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:02.818074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c0f7-687e-4fbb-891b-64e3ef47fb4b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.865559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9259-f87e-4585-9b3a-b92be411b729 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.898729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.899336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.899622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.899899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.900563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:02.900872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.902292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.910008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 249/496] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:02.910841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:02.921183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c0f7-687e-4fbb-891b-64e3ef47fb4b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:02.922750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:02.957660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:02.958290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.958578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.958861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.959508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:02.959815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:02.961195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccb1b4-88eb-4f7f-99b6-709a7e94cf99 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:02.969532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 248/497] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:02.970525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c429-91aa-4083-b19f-4dd17689e84b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:02.970808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c429-91aa-4083-b19f-4dd17689e84b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:02.971080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c429-91aa-4083-b19f-4dd17689e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:02.988535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c5f3c429-91aa-4083-b19f-4dd17689e84b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:02.989724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5f3c429-91aa-4083-b19f-4dd17689e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:02.994741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.014040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62b95-274b-4294-95bf-d276b4e02eda None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.047682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.048287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.048572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.048856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.050107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:03.050455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.051970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.056829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 249/498] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:03.058913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395454ff-c423-40c5-b5c1-26975564befc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.059258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395454ff-c423-40c5-b5c1-26975564befc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.059534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395454ff-c423-40c5-b5c1-26975564befc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.078759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-395454ff-c423-40c5-b5c1-26975564befc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:03.079943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-395454ff-c423-40c5-b5c1-26975564befc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:03.093345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 250/499] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:03.093345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.162520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 250/500] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:03.164218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395454ff-c423-40c5-b5c1-26975564befc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.196658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b36c91-b207-42c5-9333-a732a26d5ca6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.198402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.243973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.245079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.245507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.246500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.258054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.261556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.264293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.282184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395454ff-c423-40c5-b5c1-26975564befc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:03.287211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.288614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:03.289663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:03.289663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.289663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:03.291356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 251/501] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.292801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.333661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.334320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.334603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.334880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.345145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.346655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:03.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.362421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:03.363252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:03.363252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.363252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:03.363252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 251/502] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.363882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.409591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757926-8f40-4274-9396-25fe7fa53d4e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.441512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.445675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.446314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.446599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.446878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.465160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d631a-4b0e-4a10-91ba-97a6f219e7cc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.498408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.499018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.499299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.499626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.500270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:03.500579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.502006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.504869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.513933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 252/503] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:03.514860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.515788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.517332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.615767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa276-15bb-4bc7-9a3b-b32b8fc8daad None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.618938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 252/504] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:03.619757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.648819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.649445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.649750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.650025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.652602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.661291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.662795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:03.677477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.678398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:03.679229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:03.679229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.679229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:03.679229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 253/505] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.679457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.712523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.722471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6efc-77df-4c23-85ec-b0cf7bff2555 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.756679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.757281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.757564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.757864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.758501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:03.758805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.760191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.771392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 253/506] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:03.772162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.779711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500831c-569a-441f-b281-05a8079c8426 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.814291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.814915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.815731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.816013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.826213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:03.831528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.832451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:03.833246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 254/507] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.833945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.872638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01df4d-7f7c-44a9-aa6e-2a004c874db1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.882759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.906375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.906992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.907272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.907544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.915723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.917164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:03.929382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.930280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 254/508] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.931417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:03.935868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a0d9-cbae-49e5-8f38-e7943cffc04e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:03.946011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:03.970139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:03.970767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:03.971046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:03.971345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:03.981809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:03.983264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:03.996866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:03.997748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:03.998578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:03.998578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:03.998578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:03.998578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 255/509] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:03.998799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.033212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff225-20d2-4df4-a663-058634b295aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.065645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.066312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.066596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.066872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.067517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:04.067819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.069460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.081878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 255/510] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:04.082660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.101999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.105030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db6d16-06fd-459b-b6c6-797a76342748 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.138926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.139548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.139832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.140109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.147931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.149666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.152921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:04.161006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:04.162098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:04.163526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 256/511] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:04.166515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.183234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b000aa5-a430-4a6a-8a14-c2960bb08aec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.218152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.218758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.219038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.219313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.219964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:04.220267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.222243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.230477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 256/512] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:04.231239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.267606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c49b-b858-4aaa-9312-ea85b086e420 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.301665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.302241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.302527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.302805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.303530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:04.303838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.305290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.326975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1348d-c7f2-4237-b55d-ac16e3440782 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.342652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 257/513] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:04.343372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.363607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.364206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.364495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.364774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.365637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:04.365844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.366045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.367149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.368601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.380945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 257/514] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:04.381773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.444957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85eff7-05a0-4abc-97c1-c94ca3c316db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.478706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.479065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d058-a282-447b-b76a-6b59486cd50b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.479303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.479584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.479858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.480496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.482001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.504116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 258/515] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:24:04 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:04.505094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.515113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.515751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.516038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.516318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.517080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:04.517395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.519381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.540467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 258/516] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:04.541337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.594684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.601730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.631919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec97e36-44c6-4010-ad63-de57af903678 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.656555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaef1-106a-4be4-afa1-b5fc33f0fd63 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.670619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.671549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.672058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.672514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.686892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.688368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.695232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.696034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.696319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.696604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:04.707110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:04.708037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:04.708865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:04.708865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:04.708865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:04.708865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 259/517] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:04.710344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.772452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.773964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.823278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a257-a55e-42eb-a33e-0ac4a836f36f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.828903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.851357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:04.861591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:04.862946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:04.863229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:04.863499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:04.870798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:04.872512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:04.885494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 259/518] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:04.885494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:04.885748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:04.886682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 260/519] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:04.887533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:04.992518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44fdcd-81dd-4b6e-8fef-cad61bde7871 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:04.994199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1bdaa-ec68-4c5c-bdf4-c83e38ff0249 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.030413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.031031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.031315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.031591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.032159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.032769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.033056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.033330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.034571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.034888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.036277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.050750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 261/520] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.051602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.067395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.102722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.120192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.121662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.157943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c686-7afd-4f1d-a29c-c2555bc96b32 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.193529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.194431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.194808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.195149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.208058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.209594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:05.227969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:05.228898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:05.229774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:05.229774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:05.229774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:05.229774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 262/521] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:05.230001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.239852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 260/522] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:05.239852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.301692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.339019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.343758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2014af-7c58-4d18-9f41-8120b69e41e9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.351427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e79b52-4ac6-47a0-acb9-3d9886db7b2a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.386456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.387532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.388039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.388456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.388839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.389464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.389586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.389769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.390045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.390329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.392018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.397630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.399085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:05.410496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:05.411583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:05.412774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:05.417863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 261/523] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:05.417863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.418149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 263/524] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.419177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.525165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef989-40bb-4158-af2f-181ad5d1dc64 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.526867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314c8a-5109-4cb1-bda1-e6ad543ca8c3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.539893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.560092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.560719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.561003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.561283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.562355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.562950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.563232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.563502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.564128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.564430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.566387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.573638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.575119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.583386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 262/525] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.583386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:05.593048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:05.593976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:05.594841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:05.594841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:05.594841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:05.594841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 264/526] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:05.595063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.610762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.702137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e09d66-7fc1-4490-85d1-9868c3c7405c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.704405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90769b1b-525c-4f0e-8232-fa63c3cf0566 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.735422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.735965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.736245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.736519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.737816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.738133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.739573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.744518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.744925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 265/527] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.745454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.745719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.746118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.746398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.747129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.747439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.748962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.784288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 263/528] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.785497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.818439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.850164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:05.880735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43add456-86a6-41f6-8034-be9aa2eaa2a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.911057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de0a4-9771-4066-a76a-61362f8c6268 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:05.923097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.924015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.924896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.925965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.937345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.940278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:05.953147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:05.954046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:05.954868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:05.955266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:05.955266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:05.956457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:05.956457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:05.957866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:05.957866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:05.959430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:05.959998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 266/529] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:05.960567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:05.978356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 264/530] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:05.979365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.074209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25c3e3-b005-4ae8-8181-c263626be67f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.082879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.094060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b418af5-f1a4-4a05-938c-627b4c15b77f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.114383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.115345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.115766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.116204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.119430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.134491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.135104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.135388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.135684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.136852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:06.137282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.139013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.161774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 265/531] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:06.161774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.205295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.206771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.279963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8b76-3e39-4a88-b824-4649460615d7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.319610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.320660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.321076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.321480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.324263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 267/532] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:06.325172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.328822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.330329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:06.345436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:06.346899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:06.349332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 266/533] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:06.357728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.369783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.378635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.457925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5071e-d618-4479-8188-b87f6912e030 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.476816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ba7-3a56-4d96-94df-9efdb696ed4c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.499230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.500207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.500707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.501312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.502364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=44a33c23a10f4ae5a008e1d4d4973355) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:06.502900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.504740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.516328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.516328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.516328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.516328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.517014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 268/534] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/role_assignments?user.id=44a33c23a10f4ae5a008e1d4d4973355&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:06.518875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.529444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.531072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:06.547008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:06.547960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:06.549447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:06.549447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:06.549447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:06.549447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 267/535] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:06.555144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.702142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.738180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:06.743602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012ae4-93dd-47cf-ba0a-eefaf323c3e9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.767800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6adbb-7feb-4eba-b51e-5e2631ad7d3b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.788609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.789223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.789505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.790325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.791030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:06.791338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.793323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b376616-d8f4-4709-ad10-125ed6c0341f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.805969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:06.806852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.807424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.807877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.808739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:06.808912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 269/536] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:06.809387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:06.810990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88899d0f-ad6f-4335-bd4b-b670d5a45830 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:06.814224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a55d0-780d-4f6c-b9dd-8dbb0ef69cfa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.814700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a55d0-780d-4f6c-b9dd-8dbb0ef69cfa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.816546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a55d0-780d-4f6c-b9dd-8dbb0ef69cfa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.818373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 270/537] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:06.822587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a55d0-780d-4f6c-b9dd-8dbb0ef69cfa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:06.828499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 268/538] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:06.829743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:06.830034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:06.830306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:06.849594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:06.850832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:06.937538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a55d0-780d-4f6c-b9dd-8dbb0ef69cfa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:06.949734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 269/539] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:06.949734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.005558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:07.030443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.031393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.031862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.032310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.033224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=9ae494b13b5743a2b56054d2fae338fe) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:07.034309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.035959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.055630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 271/540] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/role_assignments?user.id=9ae494b13b5743a2b56054d2fae338fe&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:07.056565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b49e8-447c-4847-865e-b2e542b55cd8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.095444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.096109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.096428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.096704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.104146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.105637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:07.118081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:07.118979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:07.119817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 270/541] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:07.124922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.172419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:07.285329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8873e-1389-4d70-845f-9676dd51c82b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.327573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.328185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.328466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.328737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.329439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:07.329762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.331247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.342398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.344663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 272/542] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:07.350076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.451193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0977670-0136-4d72-9c21-189b8018c6bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.467761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:07.470525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:07.499808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5a14-444b-40c4-baba-f60b3cd4a53e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.514929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.515667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.515957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.516298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.534828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:07.543826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:07.544727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:07.545948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 271/543] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:07.552921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.571969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.572937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.573421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.573966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.575059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=0621ba7d8a164789958d1cadba19ea6e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:07.575374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.576821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.592215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 273/544] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/role_assignments?user.id=0621ba7d8a164789958d1cadba19ea6e&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:07.707562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c1a21-3c98-4b43-ae07-e77912df3f32 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.743415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.744396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.744879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.745349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.746289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:07.746795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.748454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.759456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 272/545] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:07.765459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:07.882895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b0ffa-5243-44bc-92d6-ae6cf1f06b7a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:07.919618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:07.920649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:07.921168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:07.921659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:07.933539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:07.935415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:07.942625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:07.952069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:07.953168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:07.954202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:07.954202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:07.954202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:07.955386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 274/546] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:07.962818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:08.025255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:08.128683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80170d-4f2a-4bd5-95a1-45b8de5c6e1c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:08.173011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:08.173558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.173869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.174147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:08.175353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:08.175662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:08.177045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35819626-296f-4829-9abf-68f89b7989f0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:08.195655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 273/547] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:08.201200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:08.314005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56706-a5f2-4658-a8e0-85abc7ea1712 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.314299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56706-a5f2-4658-a8e0-85abc7ea1712 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.314580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b56706-a5f2-4658-a8e0-85abc7ea1712 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:08.316736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 274/548] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:08.321872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.322161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.322442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:08.349306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:08.350525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:08.361047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615e778-f65c-4351-8951-a1b445b271f1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:08.426333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:08.426894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.427229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.427514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:08.441422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:08.442917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:08.461740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:08.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:08.464231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 275/549] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:08.468304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:08.478665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 275/550] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:24:08.570679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:08.623205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a94a9c-6d84-4192-9ead-a644d2931663 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:08.671413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:08.672030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.672311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.672581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:08.673315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:08.673625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:08.675644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:08.700113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 276/551] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:08.705474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:08.750748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:08.851970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683b18-8db9-4162-ba69-161482d59393 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:08.865226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:08.904761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:08.905378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:08.905658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:08.905956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.018452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.019842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.040803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ca855-281a-49ae-a857-8f9485cb0ed7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.068993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.075836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.111032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.113382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.113664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.113969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.129554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:09.136400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:09.137462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:09.138512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 277/552] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:09.150169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.234460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 276/553] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:09.236528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.330841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f394-906a-48b4-94c2-de00cda8272f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.351000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe598e-614b-4280-a002-d5b5e9ecc761 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.378039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.378740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.379557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.380243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.380597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.381639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:09.382624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.383886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d571b1-53ed-46cd-9c28-ee5026c7563b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.391931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.392781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.393257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.393733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.394962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 278/554] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:09.396500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e0b4-13b1-4f49-860b-1f3bb007bc94 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.396788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e0b4-13b1-4f49-860b-1f3bb007bc94 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.397469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140e0b4-13b1-4f49-860b-1f3bb007bc94 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.401800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 279/555] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:09.402732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140e0b4-13b1-4f49-860b-1f3bb007bc94 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.407972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.410835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-20533499-a917-44d1-9aa8-2f777034bcf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:09.430680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:09.431591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:09.432438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:09.432438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:09.432438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:09.433769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 277/556] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:09.435153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.435609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.436052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.454766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:09.456861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:09.458191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.537389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 278/557] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:09.538929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.558769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140e0b4-13b1-4f49-860b-1f3bb007bc94 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.624525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.625089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.625372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.625651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.640748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.642948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f9063e71-230d-414d-b981-eb28f0ffaeca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:09.663698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:09.665090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:09.665975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:09.665975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:09.665975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:09.665975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 280/558] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:09.666949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3b4a2-398d-4166-a8b0-c83d72217378 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.667156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3b4a2-398d-4166-a8b0-c83d72217378 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.667436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3b4a2-398d-4166-a8b0-c83d72217378 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.669350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 281/559] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:09.670106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3b4a2-398d-4166-a8b0-c83d72217378 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.696909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e016e-589f-40b6-8f91-b52d1fdf0c2c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.705151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.737088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.738112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.738627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.739108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.739948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:09.740439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.742086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362fa44-c29b-436e-9b19-14bcceba620c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.754600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 279/560] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:09.756273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.756770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.757229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.776259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:09.778020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:09.798292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3b4a2-398d-4166-a8b0-c83d72217378 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:09.800862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.837736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:09.838659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:09.839125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:09.839580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:09.840417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:09.840955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:09.842469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:09.857749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 282/561] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:09.857749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.863665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 280/562] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:09.864725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:09.972544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:09.979335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057bc80-4cc0-44b3-a3eb-9942d78340e0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.003692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fe65b-bc37-4b43-9d4e-7e9475662b5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.022201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.023129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.023561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.023981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.036466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:10.041677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.042708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 281/563] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.044158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.058568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.060138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.060803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.061268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.072177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.074583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:10.090801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.091692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:10.092569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:10.095478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 283/564] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.096512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.111523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.165227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af1d9f-cf40-417f-bbb8-58e6409cef94 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.199331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.199954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.200232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.200502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.207352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.208797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.212890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e79e-febd-498b-9cc9-6af5993031ed None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:10.220449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.221873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:10.223083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 282/565] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.223833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.241624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.250323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.250930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.251217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.251495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.252161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.252524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.254447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650a94-8aa2-4802-92f7-e74002363f61 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.262408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 284/566] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:10.263615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a62da-a341-4ff7-b043-198089be10cc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.263850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a62da-a341-4ff7-b043-198089be10cc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.264130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a62da-a341-4ff7-b043-198089be10cc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.265335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 285/567] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:10.266059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a62da-a341-4ff7-b043-198089be10cc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.327980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810106b-d94b-4f00-b344-fbb7e46721e5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.337299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.360853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.361466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.362302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.362581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.363329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.363637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.365148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.374770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a62da-a341-4ff7-b043-198089be10cc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.380864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 283/568] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:10.381683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.409189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.409857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.410144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.410424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.421210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:10.427085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.428050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:10.428888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 286/569] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.429569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.462394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.485502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236726-f7d1-415e-8c48-dbdb77c0a753 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.520577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.521182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.521469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.521777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.522509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.522821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.524250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4c0d0-85ee-43eb-af64-bb6ee23ed6ce None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.530854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618aec-d331-4a51-b6bb-cfb7eb8d0837 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.540893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 284/570] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:10.542166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.542522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.542867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.562128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:10.563091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.563313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:10.569622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.570291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.570577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.570851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.581941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.583403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:10.597976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.599058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:10.600132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:10.600132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:10.600132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:10.603747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 287/571] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.604527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.631941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 285/572] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:10.632767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.703451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.729613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa77dc-5f06-48a6-aaa1-b1aa62b6a818 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.744422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b82d8f-0fc1-4bc4-8910-6ecac51349da None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.766424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.767025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.767303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.767575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.768319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c9679f016e524268b81488141bfc8777) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.768623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.770097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.775346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 288/573] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/role_assignments?user.id=c9679f016e524268b81488141bfc8777&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:10.776114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.799117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.800575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.801013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.801440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.803028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.803547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.805775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.820445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 286/574] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:10.821615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.834509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.906653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ec4b1-e0d4-4af6-9bec-2f4ea71f29dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.953165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.953372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5aceb-26cb-438e-91b7-3c3526bb33ea None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:10.953455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.953455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.953455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.965291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.966788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:10.982094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:10.982540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:10.983583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:10.985303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:10.985303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:10.985303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:10.985303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 289/575] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:10.985303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:10.996113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:10.996903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:10.997192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:10.997561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:10.998140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:10.998445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:10.999807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.020310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 287/576] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:11.021947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.086315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.090333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383d95c-ab8f-431f-9c3e-8f643231d7f1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.127862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.128546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.128839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.129113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.132609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381947c-c82b-4d91-b5a2-075c6704008e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.139982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.141446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:11.156503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.157659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:11.159477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:11.159477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:11.159477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:11.162754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 290/577] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.163369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.172094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.172730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.173017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.173290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.184239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:11.189177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.190167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:11.190990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 288/578] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.191678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.243153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.274730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e8700-645f-41ff-821f-ff4fea979294 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.302741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d45240-55db-4c84-8d38-3e1a1d3a6f1c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.310303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.310916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.311196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.311470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.312108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:11.312410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.313809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.326403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 291/579] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:11.331720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.347339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.348205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.348756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.349082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.349454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.359667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.363942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:11.379666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.380616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:11.381549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 289/580] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.382807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.449958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871c5c-82f1-4752-b383-277f296490ca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.491180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.491644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ec512-65e9-493e-8913-99cb3daec8f6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.492050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.492431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.492804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.493731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:11.494829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.496222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.499352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.515227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 292/581] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:11.515227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.528580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.529117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.529400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.529684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.530372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:11.530680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.532194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.541109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 290/582] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:11.541109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.593055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.624792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a79fdf-40e9-460a-a07a-3cf3ecd32524 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.658441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902f21b-e7d9-4287-83fa-eb80c8e0e33a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.661196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.662285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.662767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.663259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.671496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.672952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:11.684962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.685920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:11.686789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:11.690813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 293/583] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.691618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.698432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.699051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.699338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.699616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.700349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:11.700663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.702144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.720180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 291/584] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:11.720896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.750390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.799737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b90aa2-cef2-4852-b4b2-931abfbbfbae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.826794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a7201-9387-45da-8e09-110a08eed24b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.831186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:11.836243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.836990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.837377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.837659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.845089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.847116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:11.858536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.864227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 294/585] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.865395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.873582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:11.874421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:11.874864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:11.875388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:11.887549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:11.889085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:11.906373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:11.907323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:11.908219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:11.908219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:11.908219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:11.908219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 292/586] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:11.909433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:11.978831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b858-c902-40c6-a865-f17dcd22dec3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:11.996578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.021859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.022800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.023091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.023375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.024068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.024375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.025691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdb037-4a79-4362-a8b8-8f6597c665c8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.025855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.043423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 295/587] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:12.043423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.067789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.068514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.068736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.069019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.081268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.082761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.095964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:12.099731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:12.100950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:12.106571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:12.106571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:12.106571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:12.106571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 293/588] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:12.106571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.153249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dedfe-2e0f-4329-9147-73f274b940df None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.187404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.188106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.188457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.188795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.189586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.190012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.191540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.209622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 296/589] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:12.210629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.219351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df9d32-d6f8-43d2-9026-29397050e437 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.250831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.257802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.258634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.259323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.259686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.260674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.261045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.262528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.283532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 294/590] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:12.283532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.318734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af2ff5-04dd-42bf-8145-84af90667b20 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.334298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.361176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.362479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.362768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.363045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.382441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=c615724d4e5944169ead59e30e0d04b2, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.383896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.395313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be90974-eab8-4ce0-98ca-a88642a81959 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.405308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 297/591] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:12 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:12.405308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.443121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.445038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.445462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.445826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.446567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.447699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.450744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.474939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 295/592] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:12.474939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.551685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.552612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850ef7b-f456-4679-8227-c0ec4979e090 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.600143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.600855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.601262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.601635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.613588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.616081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.623387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e6883-5100-4bf2-b460-1646e7c9fe24 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:12.633223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:12.634392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:12.635464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:12.635464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:12.635464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:12.636830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 298/593] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:12.637694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.657806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.675347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274425a1-1229-4b4e-be15-12853564584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.676461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274425a1-1229-4b4e-be15-12853564584e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.676917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274425a1-1229-4b4e-be15-12853564584e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.677798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274425a1-1229-4b4e-be15-12853564584e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.689152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274425a1-1229-4b4e-be15-12853564584e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.691331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274425a1-1229-4b4e-be15-12853564584e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-274425a1-1229-4b4e-be15-12853564584e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:12.703388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:12.704560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:12.705762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:12.707507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 296/594] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:12.708460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274425a1-1229-4b4e-be15-12853564584e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.761767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23dd18-0ce2-4161-ac74-a2bccdc5cc74 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.797351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.798279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.798726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.799143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.806952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.808637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.810788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274425a1-1229-4b4e-be15-12853564584e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:12.822541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:12.823642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:12.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:12.826145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.826974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 299/595] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:12.827902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.854572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.855268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.855652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.856236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.858128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.858702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.860515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:12.880728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 297/596] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:12.882163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:12.911893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:12.946608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a43d8-58e3-4b3a-87b6-2d0551e831f6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:12.982476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:12.983072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:12.983362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:12.983635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:12.984597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:12.984713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:12.986150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.000918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ccb0-dfbf-403e-b1ed-1e7a1ea4e565 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.004448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 300/597] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:13.005277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.043569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.044259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.044604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.044942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.045713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:13.046097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.047632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4847574-a3e8-4859-9ae3-a66f327ff6be None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.060133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 298/598] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:13.061384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930db567-abfb-423b-a582-ecaf8911cf4c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.061741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930db567-abfb-423b-a582-ecaf8911cf4c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.062075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930db567-abfb-423b-a582-ecaf8911cf4c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.063471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 299/599] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:13.064068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930db567-abfb-423b-a582-ecaf8911cf4c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.099527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.119661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2be7-5517-4b00-b543-ce318802e0ac None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.149890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.156349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.157058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.157409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.157776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.164743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.166335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:13.178287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:13.179303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:13.180390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:13.181192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 301/600] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:13.181895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.193843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930db567-abfb-423b-a582-ecaf8911cf4c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.234346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.235059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.235511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.235790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.249116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.250718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-aaf535a7-aa48-4e0e-8571-517731636966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:13.269397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:13.270572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:13.271447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:13.271447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:13.271447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:13.271447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 300/601] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:13.272419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.272701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.272974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.291478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:13.291789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232e8d4-cf17-4142-8425-e55cbf37ef4b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.292666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:13.327427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.328087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.328373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.328650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/44a33c23a10f4ae5a008e1d4d4973355/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.343518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=44a33c23a10f4ae5a008e1d4d4973355, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.344977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.375765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 301/603] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:13.376048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 302/603] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:13 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/44a33c23a10f4ae5a008e1d4d4973355/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:13.376470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.376763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.392591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.444670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.531724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d982f-f94b-4873-9129-b398ad235a2f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.576382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451217-4069-4693-a7a8-4a8cfc1173ec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.602420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.603391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.603684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.603959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.604888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:13.605196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.608166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.624286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 302/604] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:13.630127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.636625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.637275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.637561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.637881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.638522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:13.638985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.640366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.657736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 303/605] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:13.660232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.728164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.760858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137be17-e23b-4cfc-838b-0c910e8fb279 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.780740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:13.784340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81740e06-cb7b-4497-9444-58d1616adf45 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:13.798485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.799425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.799786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.800134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/9ae494b13b5743a2b56054d2fae338fe/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.816260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=9ae494b13b5743a2b56054d2fae338fe, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.817931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.824563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:13.825546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:13.826263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:13.826871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:13.841729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 303/606] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:13 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/9ae494b13b5743a2b56054d2fae338fe/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:24:13.841729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:13.843045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:13.845073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:13.870845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:13.871984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:13.873055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:13.876094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 304/607] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:13.882756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.017394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5261539-e9da-4c3d-b11f-f9b4928e062f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.050104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:14.058000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.058827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.059429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.059802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.072597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:14.078382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:14.079579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:14.080737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:14.082589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 304/608] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:14.088818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.123695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47241987-325f-4d5c-b334-6db33df6ae95 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.182027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.182567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.182852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.183127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.183829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:14.184140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:14.185580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:14.212622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 305/609] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:14.217192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.223961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f7a2-e522-4930-a6e8-58deb058bcb9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.257434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:14.306447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.307072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.307352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.307621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.308287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:14.308597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:14.310071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:14.325450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 305/610] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:14.330846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.352235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b11bb3-9d50-46b5-900e-3e5106f5960d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.394790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.395467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.395751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.396033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/0621ba7d8a164789958d1cadba19ea6e/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.421607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=0621ba7d8a164789958d1cadba19ea6e, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:14.423371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:14.449365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 306/611] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:14 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/0621ba7d8a164789958d1cadba19ea6e/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:14.459613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:14.524954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eb1f7-2652-42f7-9fa1-de78a068e248 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.564693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.565396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.565685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.565991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.578654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:14.580250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:14.596974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:14.598069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:14.598960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:14.598960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:14.598960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:14.599574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 306/612] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:14.607262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.699541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:14.737195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b746-d298-4555-b8e3-207961c6da15 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.776204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.776838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.777124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.777411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.778110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:14.778413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:14.779846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a9971-2e8a-42fa-84ad-777af270e0e6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:14.795960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 307/613] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:14.801246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:14.912190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:14.918472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101f840-fc02-4886-986e-ac672502b251 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:14.939320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10888-c1d7-4f1f-9a46-4f3c19adc031 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.940008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10888-c1d7-4f1f-9a46-4f3c19adc031 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.940536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10888-c1d7-4f1f-9a46-4f3c19adc031 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.942218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 308/614] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:14.960024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.960210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.960492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.979581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:14.980797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:14.981303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:14.981813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:14.991409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:14.995628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:14.999075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:15.000781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:15.025066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:15.026110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:15.027078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:15.031915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 307/615] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:15.039334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.151051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 309/616] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:24:15.195423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ac626-faae-4fb5-bba1-670731d5fe10 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:15.237651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:15.238690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:15.239185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:15.239669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:15.240594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:15.241114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:15.242865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:15.262193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 308/617] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:15.268827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.384388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:15.438125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ebca1-2fe3-4906-a3f4-57e35ef2ac40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:15.482706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:15.483347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:15.483631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:15.483907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:15.485394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.501757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:15.503194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:15.504263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:15.528036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:15.529043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:15.529941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:15.529941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:15.529941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:15.529941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 310/618] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:15.539721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.609448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83672c-9ac4-4392-b04c-5f4614f25f01 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:15.651778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:15.652785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:15.653278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:15.655340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:15.675617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:15.682937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:15.684009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:15.684820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:15.692939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 309/619] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:15.703982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.754216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc7d89-959d-4070-9704-fd292b9e6477 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:15.790091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:15.790978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:15.817654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047f64d-1efc-4d61-a220-46f322db6765 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:15.857086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:15.859075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:15.859965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:15.860530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:15.861366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:15.861953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:15.862877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:15.863385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:15.863856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:15.864380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:15.865280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:15.866987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:15.867551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:15.869153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:15.892214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 311/620] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:15.892783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 310/621] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:15.898400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:15.912565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.043865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7b31-ab6b-4f06-bbef-903e3f69c2db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.049977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef521c0-da40-447c-9261-68a4f2815a17 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.082861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.083764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.084205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.084672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.090408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.091748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.092253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.095744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.097571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.099124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.101604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.103576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-34ed6cad-10bc-4cf6-aba3-2dba2bc9a4cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:16.116178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:16.117126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:16.118029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6fc6e0b0-41da-4ec0-ac10-8e5727155ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:16.118478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:16.119319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 312/622] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6d5b-8b0b-4ad5-9a00-d368f179c461 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6d5b-8b0b-4ad5-9a00-d368f179c461 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.120208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6d5b-8b0b-4ad5-9a00-d368f179c461 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.121749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 313/623] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:16.123444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6d5b-8b0b-4ad5-9a00-d368f179c461 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.127059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 311/624] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:16.132072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.132072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.132072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.149125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:16.150523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:16.163079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.244661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 312/625] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:16.244661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.267396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6d5b-8b0b-4ad5-9a00-d368f179c461 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.344504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.345476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.345945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.346402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.347195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:16.347741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.349281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.355340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f287d9-4e9b-4069-9ae3-25f5586ebccb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.380057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 314/626] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:16.385516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.385679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.396536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.397151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.397429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.397696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.398428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:16.398732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.400160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.413401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 313/627] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:16.419028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.498743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.515889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dba8f-10b8-4dde-8bb8-1e64120f20f9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.540844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0b1a-82fa-4ed3-b5b8-8b6c480aabf6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.552750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.553500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.553898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.554249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.562158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.563715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-06f12ec2-45bb-452a-9c25-bae4a26ee76f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:16.576661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:16.577851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:16.579116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:16.579808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.579808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.584585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.584585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c9679f016e524268b81488141bfc8777/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.584972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 315/628] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:16.584972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dbd8-27da-45b4-bb32-d62d751d87fc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.584972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dbd8-27da-45b4-bb32-d62d751d87fc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.584972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25dbd8-27da-45b4-bb32-d62d751d87fc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.584972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 316/629] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:16.585687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25dbd8-27da-45b4-bb32-d62d751d87fc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.598664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=c9679f016e524268b81488141bfc8777, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.600596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18482a8-ab4c-4ec1-b298-b64b19927dfe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.619865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 314/630] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:16 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c9679f016e524268b81488141bfc8777/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:24:16.627070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.629173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.629645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.660929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:16.662678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:16.709778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.730216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25dbd8-27da-45b4-bb32-d62d751d87fc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.761234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 315/631] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:16.762423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.793746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.796377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.796819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.798598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.799578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:16.800303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:16.803213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:16.834690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 317/632] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:16.836058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:16.840580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:16.915543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe9a3b-537e-4c53-bd33-869e1501e224 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.958541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:16.959442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:16.959892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:16.960556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:16.970422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e819f-0866-4be7-92a8-aa234ecc794a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:16.974159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:16.979286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:16.980435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:16.981475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:16.983917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 316/633] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:16.990870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.040314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.041115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.041493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.041886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.058455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.146393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.148059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe354c-cab5-4e86-b523-2327dd943104 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.150159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.161223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a52ea-3d41-4643-94c5-cf9c2ea2b9b7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.198414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.199055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.199403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.199741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.200578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:17.201055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.202444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.211110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 317/634] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:17.211891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.263081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 318/635] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:17.268888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a165a-9782-409d-a98c-c5d14c0a0025 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.269182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a165a-9782-409d-a98c-c5d14c0a0025 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.269467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a165a-9782-409d-a98c-c5d14c0a0025 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.270818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 319/636] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:17.271534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a165a-9782-409d-a98c-c5d14c0a0025 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.311723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.325678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deaeed-56bf-476f-aeef-b7d916aa8d0d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.360441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.363793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.363793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.363793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.372208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:17.378092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:17.379241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:17.380132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 318/637] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:17.380941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.398437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a165a-9782-409d-a98c-c5d14c0a0025 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.400150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.448352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.449100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.449427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.449631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.467258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.469135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-741b8251-9a3f-4447-9260-a78b8b658a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:17.487427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:17.488421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:17.489330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:17.489330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:17.489330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:17.495815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c0cfe-e53e-423d-8a2c-35f1c8a15bb6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.496481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 320/638] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:17.498012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.498503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.498988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.518877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:17.520696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:17.528654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.529291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.529570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.529950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.543095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.544540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.554761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:17.561886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:17.563056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:17.564130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:17.564130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:17.564130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:17.569861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 319/639] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:17.570722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.612574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 321/640] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:17.613434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.707767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cf9e2-e6df-4884-801d-3162baa91905 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.748729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.749344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.749624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.749932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.750590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:17.750894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.753952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.755990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.773380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 320/641] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:17.774692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.794786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aadea2-49be-4545-a4da-f7ea82955523 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.835794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2508b-28a9-4042-a416-840848254caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.836922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2508b-28a9-4042-a416-840848254caf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.837360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2508b-28a9-4042-a416-840848254caf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.837794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2508b-28a9-4042-a416-840848254caf None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.838578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2508b-28a9-4042-a416-840848254caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:17.838995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2508b-28a9-4042-a416-840848254caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.840542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2508b-28a9-4042-a416-840848254caf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:17.853427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 322/642] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:17.854456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2508b-28a9-4042-a416-840848254caf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:17.913045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:17.954945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81868030-1289-47bd-8f6a-9b4390013a8b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.981502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2508b-28a9-4042-a416-840848254caf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:17.993316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:17.993954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:17.994238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:17.994510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:17.995141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:17.995437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:17.996859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.012623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 321/643] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:18.013613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.019765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.020410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.020692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.020968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.032642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.034828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.038419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:18.052310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.053341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:18.054921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:18.054921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:18.054921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:18.054921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 323/644] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.058884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.120515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3ec68-14b1-4ce2-8cb8-f47c9bc45791 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.145594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.159618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.160260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.160536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.160806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.168503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.169978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.170900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2c1f4-1b63-4192-8d5a-c96f7889ed7a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:18.182187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.183096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 322/645] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.184153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.211203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.211836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.212119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.212398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.222866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:18.228853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.229808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:18.230633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 324/646] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.231325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.289107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.298016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fc90-e009-47ce-95b6-124c10604f42 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.334828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.335453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.335732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.336001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.336098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248010b1-9b2a-4bd6-9c28-02f3468b13dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.343089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.344528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:18.355982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.356887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 323/647] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.357810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.380053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.380674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.380958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.381233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.381904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:18.382210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.383587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.396040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 325/648] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:18.396828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.413290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.474495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7beed9-05ef-4c70-9590-8842e1c7efc8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.502921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818ea96-f4b7-4754-8103-33083445ff5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.511526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.512134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.512535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.512810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.513537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:18.513878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.515352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.520978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.538741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 324/649] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:18.539658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.540175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.540783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.541064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.541337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.542039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:18.542343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.543707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.552422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 326/650] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:18.553064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.649580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.653201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d49821-dec0-4b1e-b3dc-2bf7f6b92871 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.659366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fab16-86c8-49a2-ba25-c73d0c0d65db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.691397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.692010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.692282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.692554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.693244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:18.693549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.695282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.697961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.698597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.698878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.699152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.706785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.711856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 325/651] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:18.711856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.712080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:18.724225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.725180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 327/652] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.726155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.767233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.828345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88b37-0436-4a84-8303-3b3aeed1fe71 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.832804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123ffe-466b-4042-8822-d73e2ce0bbff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:18.866851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.867809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.868247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.868527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.873697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:18.874575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:18.874998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:18.875415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:18.888760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.890249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:18.899727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:18.912703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:18.913648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:18.914560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:18.914560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:18.914560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:18.915106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 328/653] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:18.916972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:18.945133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:18.946749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.009419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.019577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3530c-cee4-4f33-a014-b4a734533202 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.046379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 326/654] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:19.047215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.052612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.053214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.053492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.053787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.054511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:19.054821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.056274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.062089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 329/655] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:18 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:19.062873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.113961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.163281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538a244-5eae-46ae-b279-a67e24e2d569 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.179599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ee583-3efd-4c7c-b568-6fdb54b3d488 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.205153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.205998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.206298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.206574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.207996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:19.208318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.209939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb02ff-37ce-4adb-a1e8-93cb8237ceb1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.222239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.222871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.223161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.223437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.224095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:19.224401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.225812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 327/656] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da1ca-1d3d-495b-8281-d4b524d3a004 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da1ca-1d3d-495b-8281-d4b524d3a004 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da1ca-1d3d-495b-8281-d4b524d3a004 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 328/657] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:19.239112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da1ca-1d3d-495b-8281-d4b524d3a004 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.239784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 330/658] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:19.243202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.260980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.356726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7b22-64ef-4c7a-8a1b-cbe770f6cbc0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.366383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da1ca-1d3d-495b-8281-d4b524d3a004 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.385968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.395649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.396258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.396537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.396813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.404622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.405485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.405870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.406152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.416868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.418358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3f44db64-93f3-4c1c-9c38-3f7193539eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:19.433387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:19.434358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:19.435244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:19.435244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:19.435244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:19.435244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 329/659] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:19.435747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.435935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.436214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.454722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:19.456478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:19.472853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.474383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.490399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.532309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 330/660] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:19.533493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.594572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 331/661] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:19.595745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.652318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.655843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6a10-f543-44a5-bc70-9b97c1e3845e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.696975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.698018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.698516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.699109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.707686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.709561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.718777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68478773-f3db-4346-b1f2-88c7b3c23633 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:19.723249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:19.724163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 15:24:19.725077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:19.732027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 331/662] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:19.733204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.733639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.754683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.755312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.755600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.755840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.756481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:19.756788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.758208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.770486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 332/663] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:19.771403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.850095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d7ac-f46b-4763-9baf-8cdc15bfdd12 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.874326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2200604-8c01-4445-916a-8f2534e6da08 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:19.884049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.884685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.884964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.885236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.896627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.898099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:19.912364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:19.913300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:19.914852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:19.914852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:19.914852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:19.914852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 332/664] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:19.914852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.919773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:19.920396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:19.920681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:19.920958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:19.921691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:19.922044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:19.923608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:19.931061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:19.948161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 333/665] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:19.948161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:19.982734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.031661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287b59-46b6-42db-b3a1-9ff2bd50712e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.057281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60bcdf-f72f-4e04-b2e2-c42baddc359e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.068194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.068830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.069116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.069646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.076296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.077760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:20.089021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:20.089918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:20.090799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 333/666] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:20.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.098149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.098789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.099075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.099348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.109587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:20.114981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:20.115869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:20.116729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 334/667] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:20.117420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.168741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.202761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30093534-7f1c-41ff-8578-24f2962483c6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.208221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.217284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba623a00-5b51-45b5-8646-a947abb8062d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.240151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.240763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.241158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.241431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.242277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:20.242588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.244071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.252805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.253407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.253692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.253991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.257307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 334/668] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:20.258415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.326529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.327984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.363107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccfe5-54f5-4b0c-bb5f-048d8126e359 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.396305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.403214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.403819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.404097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.404370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.405059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:20.405363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.406819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.420000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 335/669] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:20.420730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.431712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.443567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 335/670] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:20.444699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.534339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc8242-d37a-4e1e-acc3-f04b07cc59a7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.559368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc234-24df-4f22-9ed2-c97e12e5dc9e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.568140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.568753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.569045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.569580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.570277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:20.570585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.571961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.580012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 336/671] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:20.581344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.603094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.603733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.604015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.604291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.611064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.612547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:20.623826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:20.624687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 336/672] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:20.626533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.646362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.686456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.696142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3bd14-6f35-4f22-9863-3c240a8a530b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.731500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.732099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.732385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.732659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.733373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=44a33c23a10f4ae5a008e1d4d4973355) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:20.733678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.735137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.749727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 337/673] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/role_assignments?user.id=44a33c23a10f4ae5a008e1d4d4973355&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:20.749727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.750008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be4e4e-f959-42a4-8444-6a374381c242 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.799800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.800599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.800943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.801282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.813799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.816058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:20.833452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:20.834709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:20.835750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:20.835750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:20.835750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:20.836564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 337/674] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:20.837388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.886625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb75f2-9b94-4bea-8054-d5d1dba3b139 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.907094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:20.928748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:20.929491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:20.929881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:20.930229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:20.946937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:20.948330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:20.968844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:20.970105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:20.971218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:20.971218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:20.971218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:20.971427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27138d-2a6d-4886-8b2b-f59e1c9744e1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:20.971847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 338/675] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:20.971847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:20.974246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.014023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.014780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.015063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.015334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.016103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.016389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.018628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.031144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 338/676] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.032876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.098218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bfee0-d31c-48bb-8a06-7d0a7cbae662 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.140827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.141716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.141838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.142114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.142789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.143098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.144484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.155210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9181c3d-bfff-4b17-9919-95621c9324dc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.164359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 339/677] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.165253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.193666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.194910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.195345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.195771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.196591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.197140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.198790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.208271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.214003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 339/678] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.215238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.247275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.310594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a0ed-a84a-4ede-8f2f-6d4dc48f2d96 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.331820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8252d-b7a6-42c1-93b3-79bd7cbed9ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.352906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.353519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.353836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.354112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.355305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=9ae494b13b5743a2b56054d2fae338fe) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.355629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.357392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.372957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 340/679] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/role_assignments?user.id=9ae494b13b5743a2b56054d2fae338fe&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.373899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.378248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.378899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.379178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.379446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.387821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.389585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3d98adf4-f7a1-443d-bba8-f7e53253fd37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:21.404198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:21.405339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:21.406491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:21.410488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 340/680] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:21.412184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac098f-e987-4119-ac8e-676641b11014 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.412470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac098f-e987-4119-ac8e-676641b11014 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.412743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac098f-e987-4119-ac8e-676641b11014 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.419104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 341/681] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:21.419104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac098f-e987-4119-ac8e-676641b11014 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.495492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.504505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8c872-b12c-4331-841f-d96d8f59c35c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.540457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.544241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.545093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.545445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.545907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.556235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.557975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.563979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac098f-e987-4119-ac8e-676641b11014 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b0a1bdc7-f590-47ed-91d7-bcea2787f096 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:21.572649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:21.573649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:21.574629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 341/682] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:21.576561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.576913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.577280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.600878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:21.602395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:21.615054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.615632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.615921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.616381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.616919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.617228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.618778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.636444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 342/683] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.637369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.700881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 342/684] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:21.702171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.767181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8bafa-36b6-41ff-bc49-cbcd33fc10e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.832904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.835905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.836549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.836833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.837201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.838200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.838507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.839940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.869674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e044-5ab8-4fd1-b890-f261baebd81f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:21.873118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 343/685] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:21.878305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:21.884417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:21.919280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:21.919902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:21.920189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:21.920466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:21.921203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=0621ba7d8a164789958d1cadba19ea6e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:21.921516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:21.922987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:21.931955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 343/686] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/role_assignments?user.id=0621ba7d8a164789958d1cadba19ea6e&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:22.006426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.024618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7678-3488-4c4d-91d0-8b2c1c0652ca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.069597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.070498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.070957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.071381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.149366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.151453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.154078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:22.262575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4d0c-6963-4303-86b3-77f7772d447f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.264948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 344/687] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 15:24:22.270864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.313513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.314126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.314526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.315401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.326941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:22.334062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:22.334995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:22.335811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:22.336647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 344/688] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:22.343416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.357063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:22.403191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989417d-bd42-4cf7-bcc8-279b5752cdb8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.404953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:22.446919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.447559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.447843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.448118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.460468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.462160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.478475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babebb6-8557-4621-b4c9-fac7482cf35a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:22.478857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:22.479740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:22.481016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:22.481016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:22.481016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:22.481016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 345/689] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:22.486017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.519140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.519757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.520190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.520469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.521446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:22.522044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.523434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.534817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 345/690] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:22.541567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.632871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8421-648c-4596-a0f7-238a023f410b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.651027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:22.689377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178ac34-c476-4906-bc86-bef14f79a1d7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.720210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.720933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.721220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.721496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.722174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:22.722482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.723860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.753663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 346/691] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:22.759056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.770426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.771082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.771372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.771650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.775992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:22.798782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.800312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:22.835949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:22.837049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:22.837971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:22.837971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:22.837971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:22.846265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 346/692] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:22.851723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:22.876619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508a7ca-c172-4dc4-ad87-7cae23096ba1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:22.918433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:22.919520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:22.919811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:22.920090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:22.928462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:22.929946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:22.943467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:22.945826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:22.947931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 347/693] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:22.951071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.063466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace06d-2a18-44f7-b5ca-d1f64346984c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.078761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:23.109413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:23.126872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9cd2-08da-4386-bfc4-5b2419813ae5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.133323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.133979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.134267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.134545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.135207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:23.135518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.136908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:23.150230 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 347/694] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:23.159485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.189935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.190469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.190747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.191017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.191710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:23.192013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.193438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:23.219822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 348/695] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:23.223875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.322927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ce5b8-01e1-4e7b-a682-3432bddb20ff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.354342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55559-282d-4855-a37b-a33b957adcac None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.361242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.361957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.362286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.362563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.375060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.376669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d67aa840-5e4c-487c-b9a1-e5084aac3635 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:23.389831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:23.390801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:23.391718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:23.394753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 348/696] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:23.396214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c7eec-7490-4b7b-85ce-adaa9fb31061 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.396697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c7eec-7490-4b7b-85ce-adaa9fb31061 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.397193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c7eec-7490-4b7b-85ce-adaa9fb31061 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.399051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 349/697] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:23.399903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c7eec-7490-4b7b-85ce-adaa9fb31061 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.405291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.406196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.406679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.407150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.408089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c9679f016e524268b81488141bfc8777) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:23.408650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.410400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ef717-18ce-439e-8eed-44062b00cee4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:23.417131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 349/698] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:23 2022] GET /identity/v3/role_assignments?user.id=c9679f016e524268b81488141bfc8777&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:23.417405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:23.422293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.422882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.423419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.459188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:23.462426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:23.488107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:23.600266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c7eec-7490-4b7b-85ce-adaa9fb31061 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.604036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 350/699] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:23.647948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.648625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.648913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.649243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.650018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:23.650379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.651836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:23.672317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 350/700] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:23.678465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.786723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2013ca5-1627-4e3e-a819-afc370dea53b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:23.815859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:23.825547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:23.826455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:23.826940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:23.827433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:23.893817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:23.906203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:23.907940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.038887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.043112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae697-6260-4000-9c3d-e213f81631df None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.061835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 351/701] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:24.069811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.099756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.100496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.100810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.101089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.101778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.102373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.104183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ea51152c-231d-4866-9905-f659d136ca23 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:24.117531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:24.118710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:24.119588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 351/702] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Sep 12 15:24:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:24.120403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0c43-14bd-433a-831f-61c51fd49a4b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.120989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0c43-14bd-433a-831f-61c51fd49a4b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.120989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0c43-14bd-433a-831f-61c51fd49a4b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.123245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 352/703] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:24.128028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac0c43-14bd-433a-831f-61c51fd49a4b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.200780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8864c-c732-4ed5-a759-4c2e0451875e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.243548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.244329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.244645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.244981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.256378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.256653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac0c43-14bd-433a-831f-61c51fd49a4b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.258411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-dd79daa7-5b44-4703-bafd-1f7ca187a9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:24.275910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:24.278186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 352/704] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.280326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.293973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.294998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.295286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.295853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.296497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:24.296806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.298235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.300324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:24.301716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:24.308354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 353/705] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:24.309173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.342458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.394033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 353/706] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:24.395282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.417077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.432726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7aa3-fc23-4770-b13d-4b39d5c308af None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.470413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.471877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.472309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.472755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.473838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:24.474295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.475825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.489607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 354/707] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:24.494586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.594496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9328464-3976-4aa3-9791-2010ac6ed753 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.605947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.618930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be35d0f-416d-4173-bbb0-960f467ad999 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.654903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.655525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.655806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.656078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.656816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:24.657130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.659368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.664567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.665458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.665657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 354/708] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:24.666132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.666413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.685677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.687226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:24.707508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:24.708469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:24.709343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 355/709] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:24.714644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.818379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.846705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:24.874702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090319-b495-4412-b047-cde8d0090652 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.918107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.919175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.919465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.919745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.921039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:24.921039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:24.922203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6e78e-3f1e-4286-8662-cc096534812f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:24.936737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 355/710] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:24.938375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf17c2-a7e9-439a-9948-3327576c2047 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.938844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf17c2-a7e9-439a-9948-3327576c2047 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.939300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf17c2-a7e9-439a-9948-3327576c2047 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:24.941468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 356/711] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:24.946065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0c11-309b-4ea1-b264-a2823cc4e817 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:24.946609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf17c2-a7e9-439a-9948-3327576c2047 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:24.988831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:24.989471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:24.989832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:24.990109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.003824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.005927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-63993839-aa3b-44a0-8402-833c227ad7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:25.025260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:25.026263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:25.027137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:25.027137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:25.027137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:25.028118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:25.029782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 356/712] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:25.031467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.031996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.032276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.051583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:25.052723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:25.073690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf17c2-a7e9-439a-9948-3327576c2047 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:25.111541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:25.113327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.113802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.114472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.115500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:25.115963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.118168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.120242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:25.121211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 357/713] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:25.122893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:25.134545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 357/714] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:24 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:25.326753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b01b0-daf1-4b08-9c96-c8886f44bbf4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:25.360473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:25.366809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:25.367389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.367664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.367935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.368583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:25.369072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.370745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.385548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 358/715] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:25.394567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:25.414225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:25.576021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b9d18-21ec-4f38-b625-1f5af7f84e6b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:25.582186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec116e7-2778-4663-a7bb-a8aae5afcb6e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:25.640209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:25.642119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.643588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.644950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.645767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.647702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.648291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:25.649160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.649641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.650137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.651370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.653251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:25.670366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:25.671434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:25.672291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:25.676377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 359/716] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:25 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:25.682415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:25.704056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 358/717] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:24:25 2022] POST /identity/v3/users => generated 254 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:25.766613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:25.825602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccf956-9e9e-4b8a-8402-d45d050a84d7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:25.866258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:25.866892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:25.867175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:25.867453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:25.868097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:25.868402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:25.869809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4567f-2529-4d6b-b361-64c48ca956fb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:25.874971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 360/718] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:25.880394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:25.988257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:26.043840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59bf3-dd0d-4b18-bdd3-da9602fcbf80 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:26.082676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:26.083298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:26.083580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:26.083854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:26.084497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:26.084805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:26.086248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c3225-3684-4fd0-bf25-619a0ab072e8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:26.091384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 359/719] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:26.295083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5727a-40fe-46a3-897f-fb5b823187c8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:26.295442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5727a-40fe-46a3-897f-fb5b823187c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:26.295662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5727a-40fe-46a3-897f-fb5b823187c8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:26.297237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 361/720] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:26.302651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:26.303097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:26.303268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:26.323508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:26.324755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:26.403419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 360/721] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:26.766668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5727a-40fe-46a3-897f-fb5b823187c8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:26.900767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5727a-40fe-46a3-897f-fb5b823187c8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:26.928630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:26.952240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:26.952878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:26.953164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:26.953444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:26.969499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-489632c6-9c38-4690-9b06-1a694db2d7bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:26.974624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:26.975605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:26.976862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 362/722] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:26.982240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.036010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:27.090937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa8463-09cb-476a-8003-ed355a404b46 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:27.122988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2eba-004c-4acb-a8c2-fe2965449c0c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.123311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2eba-004c-4acb-a8c2-fe2965449c0c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.123551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2eba-004c-4acb-a8c2-fe2965449c0c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.124879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 363/723] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:27.130794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.131080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.131358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.138795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:27.139421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.140036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.140036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.140631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:27.140939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:27.142352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:27.153570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 361/724] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:27.166171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.172650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:27.173955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:27.263920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 364/725] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:27.318532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6d4c3-5707-4cb3-8244-8d677bbfcfdd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:27.334544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:27.360197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:27.360830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.361117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.362036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.376721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:27.378954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:27.393075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:27.394042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:27.394922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:27.394922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:27.394922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:27.394922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 362/726] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:27.400105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.513298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b82a7c-6cfb-494e-bd14-df26171d4528 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:27.530066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:27.550609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:27.551225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.551512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.551793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.552442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:27.552750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:27.554159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:27.566864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 365/727] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:27.572606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.670757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.683498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb9a5c-43b8-4da7-b320-f4cd67da5803 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:27.751380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:27.752132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.752447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.752727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.771998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:27.773449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:27.800839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:27.801873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:27.802812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 363/728] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:27.807815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:27.863933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217fdd2-de6d-4dfb-b208-f830e8dd6fee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:27.894071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:27.905338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:27.906548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.906832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.907105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:27.907766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:27.909172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6676a-edbe-44fa-880e-da9c2fdc8e43 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:27.944747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 366/729] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:27 2022] POST /identity/v3/services => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:24:27.998953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf74c9-3818-45cf-b682-b9125ea12584 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:27.999238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf74c9-3818-45cf-b682-b9125ea12584 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:27.999573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf74c9-3818-45cf-b682-b9125ea12584 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.000928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 367/730] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:28.006506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6339d8-726c-4e50-810f-9024cc7e7469 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.008312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.008596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.008869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.031481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:28.033465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:28.062133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.122591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.123169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.123476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.123702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.124452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:28.124762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:28.126295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:28.129488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 368/731] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 15:24:28.147253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 364/732] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:28.152867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.255998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0232a4-eba7-4eba-b237-31e4ae338997 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.291122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.291736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.292023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.292301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.322633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.363636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:28.365110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:28.441292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.482622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 369/733] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 15:24:28.491860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.537012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.566417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.588549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a10dc5-4124-44e4-875c-72f512027d57 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.619198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558052-b436-494b-b569-7b572bae41d0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.654927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.655497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.655782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.656060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.660280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.660919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.661205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.661473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.679884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:28.681659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:28.683203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:28.692833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:28.693834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:28.695337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 365/734] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:28.700621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:28.707183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:28.708169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:28.709136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:28.709136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:28.709136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:28.711486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 370/735] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:28.723017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.842611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efdec0-aaae-4fa0-bf5e-60a9d13f16d7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.854420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.860536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b613-1193-4f42-a066-9e068ad552fe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:28.881093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.881792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.882017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.882299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.882945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:28.883254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:28.884644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939327d-53b2-4a31-a105-93fe8ce85878 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:28.897674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 371/736] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:28.898954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0ed4-4b61-4562-b57d-be964bfcde2b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.899246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0ed4-4b61-4562-b57d-be964bfcde2b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.899526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca0ed4-4b61-4562-b57d-be964bfcde2b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.900972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 372/737] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:28.902715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca0ed4-4b61-4562-b57d-be964bfcde2b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:28.902919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:28.903668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.903954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.904232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.905027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:28.905283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:28.906710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8f6c-88c1-4f67-bb01-3744f91f7615 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:28.910011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:28.916767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 366/738] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:28.918450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:28.918844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:28.919223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:28.938440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:28.939834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:29.010537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 367/739] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:29.011559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.027378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca0ed4-4b61-4562-b57d-be964bfcde2b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.066730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.067576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.067965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.068448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.076277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.077901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f116278e-e16e-4b44-be07-f535c5277a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:29.091269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:29.092233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:29.093283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 373/740] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:29.096839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8b3e-5c9a-4513-b558-dfe51ebe7716 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.096839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8b3e-5c9a-4513-b558-dfe51ebe7716 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.096839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe8b3e-5c9a-4513-b558-dfe51ebe7716 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.096839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 374/741] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:29.098462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe8b3e-5c9a-4513-b558-dfe51ebe7716 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.130657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.145643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e453d4-23e1-4e81-b8aa-1ba434367c9e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.182928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.193744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.194882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.195181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.195456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.208930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.211356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.222127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe8b3e-5c9a-4513-b558-dfe51ebe7716 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-626545ab-23a1-4657-9ccc-f0e44d5b5a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:29.227987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:29.228920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:29.229934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:29.229934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:29.229934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:29.231646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 368/742] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:29.233100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.233565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.234081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.253505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:29.255082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:29.262717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.263383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.264214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.264504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.265203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:29.265566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.267620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.290317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 375/743] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:29.290317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.339091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 369/744] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 15:24:29.342575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.385529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.403561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1582d3-b2ba-430c-815a-bb91473766b2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.450933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.452201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.452710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.453239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.454652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:29.455219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.457413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.467007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa16c8-85e0-46dc-b96f-b594d1bf4f83 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.477927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 376/745] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:29.479506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.480654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.506656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.507268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.507550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.507831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.508476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.509902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.528496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 370/746] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:29 2022] POST /identity/v3/services => generated 240 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:29.529458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.613392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711f3f6-24b0-4977-871d-e54cc564a8a1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.660609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.661627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.662144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.662621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.663552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c9679f016e524268b81488141bfc8777) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:29.664083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.665787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.672472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 377/747] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/role_assignments?user.id=c9679f016e524268b81488141bfc8777&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:29.672757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f619-501d-498f-bc69-a7f57223383e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.715334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.716129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.716237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.716444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.717622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.727780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.729352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.735805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:29.749746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:29.750861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:29.751810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 371/748] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:29.755862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.822982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:29.903460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25022dc8-ebaa-4d67-a30f-ca0581e9ce55 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.929910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd059eb-85f4-4554-9399-48e96ce31b2a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:29.951836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.952484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.953070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.953355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.967319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:29.973005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:29.973623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:29.973854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:29.974729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:29.975575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:29.976700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:29.976700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:29.976700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:29.976700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:29.976966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 378/749] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:29.977034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:29.982264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:29.999307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 372/750] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:30.005897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.082075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.134717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b8bc-719a-4e64-a44a-5d983dd1911d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.157935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.171885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.172495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.172784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.173053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.173729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:30.174143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.175662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.180643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e527d5-35a7-4258-8fea-4ea84d125d60 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.184772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 379/751] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:30.190519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.257448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.258145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.258430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.258704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.279842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.281412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.297976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957426-3986-4b2b-ae47-346dd82dd9eb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:30.301577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:30.303463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:30.304558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:30.304558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:30.304558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:30.304558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 373/752] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:30.310712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.329088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.335842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.336535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.336821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.337094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.348800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.350315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:30.368905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:30.371191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:30.373267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:30.373267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:30.373267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:30.373267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 380/753] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:30.380526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.516169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f245da2-baf5-4bdd-9f8c-a2c2037fe8d1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.585132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02786b-e5ea-4846-8c96-8694c51cf234 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.590246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.592042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.592628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.602179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.602886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:30.603203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.604643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.624566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.629211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 374/754] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:30.634143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.635065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.635207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.635803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.636320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.637201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:30.637754 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.639478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.678533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 381/755] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:30.683581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.750332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d1621-2ef2-4a43-8cef-3317ce83c4d0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.765045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.788616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.789416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.789813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.790157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.796855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.798429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0890ffc3-70a0-467b-ac35-ecf22a74c91b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:30.810285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:30.811157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:30.812124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 375/756] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:30.813443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8898-4064-4d0a-97af-0ebd2da9f89a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.813840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8898-4064-4d0a-97af-0ebd2da9f89a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.814180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8898-4064-4d0a-97af-0ebd2da9f89a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.816284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 376/757] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:30.817650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb8898-4064-4d0a-97af-0ebd2da9f89a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:30.842394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0cf95-83ad-4281-957e-f56c6aefde47 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.889087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.889756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.890049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.890323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.899586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.901045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.906275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-251c3fb1-c9e5-4be3-a82a-d717d505b2b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:30.916332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:30.917901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:30.919117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 382/758] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:30.919847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.919847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.919995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.933633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb8898-4064-4d0a-97af-0ebd2da9f89a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:30.942376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:30.944026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:30.969097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:30.969933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:30.970330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:30.971309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:30.972220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:30.972626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:30.974406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:30.993833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 377/759] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:30.994999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.024369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 383/760] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:31.025206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.058458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:31.138408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051cc6a-b770-4914-9915-58ad4453f660 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:31.178129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:31.179179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:31.179616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:31.179898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:31.181851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=c615724d4e5944169ead59e30e0d04b2, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:31.183443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:31.183861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c316bf-aa27-4984-8d86-c2a2fafb9466 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:31.201069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 384/761] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:31 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:31.234181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:31.260561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:31.261187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:31.261466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:31.261835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:31.262491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:31.262795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:31.264286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:31.291140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 378/762] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:31.296328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.438074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:31.445757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e744c-275b-41ea-b7bc-ad6e6116c3fd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:31.484518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:31.485234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:31.485527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:31.485842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:31.510222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.557346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:31.558779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:31.665219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 385/763] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:24:31.665489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:31.671762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.736804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998155-3362-469f-aa62-690a87e1b2ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:31.825002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:31.826044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:31.826554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:31.827055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:31.827850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:31.829443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:31.844282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:31.845302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:31.846203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:31.855430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 379/764] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Sep 12 15:24:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:31.861725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:31.881377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c5cc-2c20-417d-a4ac-a35e0b876eaf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:31.932210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:31.946427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:31.947239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:31.947526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:31.948127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:31.969184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:31.972141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:32.001307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:32.002921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:32.003824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:32.003824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:32.003824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:32.008696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 386/765] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:32.009552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.036712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fb8d9-e402-49e8-85b9-a78395ffff0e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.097146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.106220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.107147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.107674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.108242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.109083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:32.109594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.111734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.125578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 380/766] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:32.137730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.192168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc32f-c22c-4f26-ac91-63c55c5eb2bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.234523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.235417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.235705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.236060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.236795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:32.237199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.238625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a303aed-db49-4efc-ba9d-b2efcbae3a31 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.253508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 387/767] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:32.254439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa68fd6-c5dc-41ba-beff-42ab28754c68 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.254886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2bdf-a68b-4cc4-a012-191bb20990d6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.255313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2bdf-a68b-4cc4-a012-191bb20990d6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.255595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2bdf-a68b-4cc4-a012-191bb20990d6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.261624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 388/768] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:32.262609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2bdf-a68b-4cc4-a012-191bb20990d6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.262983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.304162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.304770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.305049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.305321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.306150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:32.306516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.307924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d580e-e36b-41f9-97b9-edc155410fca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.313120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 381/769] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:32.314159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5fea-23cd-46b0-a3f0-79b8a04b529e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.314998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5fea-23cd-46b0-a3f0-79b8a04b529e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.315278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5fea-23cd-46b0-a3f0-79b8a04b529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.333642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ceaa5fea-23cd-46b0-a3f0-79b8a04b529e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:32.334887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ceaa5fea-23cd-46b0-a3f0-79b8a04b529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:32.383086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2bdf-a68b-4cc4-a012-191bb20990d6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.384758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 382/770] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b2ee-879b-4824-a9f1-ecb9f2f99055 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b2ee-879b-4824-a9f1-ecb9f2f99055 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b2ee-879b-4824-a9f1-ecb9f2f99055 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 383/771] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:32.421814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8b2ee-879b-4824-a9f1-ecb9f2f99055 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.431422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.432304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.432742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.433138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.441384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.443099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-05fc2212-853a-4ca3-901f-99bb077c8555 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:32.456143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:32.457196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:32.458105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:32.458519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 389/772] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:32.459721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.460070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.460417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.478773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:32.480191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:32.533818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.540182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8b2ee-879b-4824-a9f1-ecb9f2f99055 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.565459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 390/773] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:32.566426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.606033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.606664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.606950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.607226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.607994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] RBAC: Adding query filter params (interface=public, service_id=cea87ebc28b94a698eacadd1a8cd7bdc, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:32.608304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.609747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.617682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 384/774] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/endpoints?service_id=cea87ebc28b94a698eacadd1a8cd7bdc&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:32.623033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.715661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.730468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f83aa-25db-448f-88b3-f7adf302e314 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.768354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.768972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.769254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.769528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.770367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:32.770717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.772153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.773914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee18ecc-a4de-4772-aee0-123e710de0fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.784602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 391/775] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:32.785489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.826031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.827009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.827486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.827974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.829320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:32.831530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:32.839435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 385/776] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:32.840412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:32.877598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:32.935367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e42366-9b44-494f-a4a8-ea01101f7868 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.978962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:32.979600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e9cb1-db16-47cd-8703-f795db9dce7f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:32.979686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:32.979918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:32.980073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:32.994299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:33.001183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:33.002765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:33.003610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 392/777] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:33.004322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.028546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.035279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.035898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.036178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.036448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.037676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=b232365f81bc4933bec7a8a86778a494) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.038029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.039810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.047264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 386/778] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:32 2022] GET /identity/v3/role_assignments?user.id=b232365f81bc4933bec7a8a86778a494&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:33.047977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.149961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011f32-8eff-4b1b-ad6c-dc89a182fa64 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.169414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.181729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9797c5-a3fc-4546-87da-2ee39d93839d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.194217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.195449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.195937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.196633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.198024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.200056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:33.211503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:33.212647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:33.213478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:33.216430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 393/779] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:33.222785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.229753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.230374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.230656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.230928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.232364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.232670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.239157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.255178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 387/780] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:33.262357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.350517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.394814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdcab-dcc9-4e1e-93cf-912d2c31b6f6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.409423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b774b2c-252c-4ebd-b9be-07829d77cc10 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.464860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.465482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.465786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.466065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.466697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.467000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.468396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.476905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.477546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.477854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.478130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.481449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 394/781] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:33.489452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.495020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.496556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:33.526408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:33.527407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:33.529398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:33.529398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:33.529398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:33.529398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 388/782] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:33.534426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.547812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.609843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc476c-f5cd-417f-be0f-6622e2be495b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.651550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.652165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.652447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.652720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.653298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae841989-72a0-4c93-a641-2000f85d1834 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.653469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.653635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.655309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561ae35-f9a9-43c0-a435-0c64bd488e67 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.663030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 395/783] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:33.664715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fadefb-9dd6-4e6f-b3b9-7d981b2145a6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.665412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fadefb-9dd6-4e6f-b3b9-7d981b2145a6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.665698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fadefb-9dd6-4e6f-b3b9-7d981b2145a6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.667688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 396/784] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:33.668405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fadefb-9dd6-4e6f-b3b9-7d981b2145a6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.673449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.691513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.692250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.692601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.692969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.693684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.694179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.695912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbd44-eafc-4b47-b606-1b285c10dc91 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.709297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 389/785] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:33.714778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.715145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.715503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.734557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:33.735805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:33.793828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fadefb-9dd6-4e6f-b3b9-7d981b2145a6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.801696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:33.810322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 390/786] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:33.811364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.840266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:33.841324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:33.841689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:33.842005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:33.843039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] RBAC: Adding query filter params (interface=public, service_id=80e7d6e8f3924c8a8bdf678dfebedfaa, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:33.843358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:33.845874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:33.855840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 397/787] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/endpoints?service_id=80e7d6e8f3924c8a8bdf678dfebedfaa&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:33.861492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:33.981473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f42d1-e288-47f7-916b-692002ecb1fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:33.998123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:34.023408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.024055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.024339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.024615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.032585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.034169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.046992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cbd09-9ad6-48e7-a3d2-ebda165f662c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:34.047644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:34.048523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:34.049667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 391/788] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:34.058499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.096067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.097187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.098418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.099074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.101341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.102981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.113298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 398/789] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:33 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:34.115201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.181315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:34.291411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d9ae3-a3e1-4dae-be80-23175db711b0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.315636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b807b-e9da-47bd-84ca-72420bf5bea2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.349277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.350833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.351301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.351757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.353101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:34.353965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.355738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.355848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:34.363102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.363743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.364022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.364295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.378316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:34.383029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 392/790] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:34.386435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:34.387348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:34.388190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:34.388822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.390609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 399/791] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:34.402666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.633577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d93d3-d556-4257-897e-8d6695b9f30b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.636641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460d67-2a55-4709-981f-28c6de27f25a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.671070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:34.677913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.678529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.678810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.679083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.679802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:34.680111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.681508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.691907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 400/792] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:34.697652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.735311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.735963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.736308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.737090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.756240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.757858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.758425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:34.777536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:34.778564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:34.779628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:34.779628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:34.779628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:34.779628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 393/793] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:34.787538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.812806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d2d6-fcb9-4fdb-9ef8-7e0cb4cdac23 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.853273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:34.853933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:34.854219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:34.854495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:34.866144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:34.867609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:34.882683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:34.883709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:34.885195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:34.885195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:34.885195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:34.885195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 401/794] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:34.893741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:34.990225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd1508-a887-4174-8ec8-cf7066f6d15c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:34.991165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119879-b063-408c-bb51-d674c23bd908 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.014078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.026333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.026990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.027362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.027647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.032894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:35.032894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.032894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.042921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 402/795] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:35.049727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.067883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.068850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.069591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.070111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.070963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:35.071467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.073033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.104272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 394/796] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:35.120835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.156258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2dfd2-a198-4086-a2eb-9756cb7976a5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.180667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.197729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.197729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.197729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.197729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.202288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.203742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:35.216333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:35.217296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:35.218820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 403/797] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:35.224204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.275679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.333645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1e06-0386-4535-bf41-5dc8431a25e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.334948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d780-f676-4b57-aa66-e2faf51ee013 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.370510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.371124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.371471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.371746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.372523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:35.372836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.377511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.390433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.391095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.391381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.391660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.397834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 404/798] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:35.399372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.412164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.413578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:35.442108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:35.443062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:35.443958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 395/799] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:35.454693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.506136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.512163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4930d-9394-4a4f-bbd0-2ff3327d7cd3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.549472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.549927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.550042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.550238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.611366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.629422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8ac34-71dd-48e1-a0ec-9b8a27aae6b4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.635834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.637564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866cce4-1022-4f08-a466-4e2cac276f13 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.676827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.677471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.677885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.678235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.679069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:35.679427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.682474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ae46e-eeb8-4ff2-ba84-512e00118db5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.700767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 396/800] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:35.702999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3d849-220c-45d1-be18-8a29b9b76da4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.703580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3d849-220c-45d1-be18-8a29b9b76da4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.704057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3d849-220c-45d1-be18-8a29b9b76da4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.711381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 397/801] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:35.711381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3d849-220c-45d1-be18-8a29b9b76da4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.746445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.769627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 405/802] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 15:24:35.771865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.772650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.773137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.795045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:35.796389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:35.823175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3d849-220c-45d1-be18-8a29b9b76da4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:35.860137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:35.860800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.861081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.861571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c9679f016e524268b81488141bfc8777/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.863382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=c9679f016e524268b81488141bfc8777, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:35.865306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd389-452f-4d82-b2ca-501f1b3cf61f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:35.871939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 406/803] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:35.885731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 398/804] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:35 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c9679f016e524268b81488141bfc8777/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:35.885731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3823fc-98b1-4bdd-b0be-524f03b56057 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.885731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3823fc-98b1-4bdd-b0be-524f03b56057 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.885731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3823fc-98b1-4bdd-b0be-524f03b56057 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.886131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:35.886206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 399/805] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:35.903418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:35.903726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:35.903970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:35.919579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:35.936053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:35.937230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:36.089417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278e33-292f-46f8-8299-c72070d5fe6d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.095033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 400/806] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:36.113911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:36.144658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.145290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.145571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.145880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.159687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.161025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.162543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:36.180120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:36.181075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:36.181990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:36.181990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:36.181990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:36.183212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 407/807] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:36.188450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.368608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e453a-2884-43bb-9f02-4c55289d1b2d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.428824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.429813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.430098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.430376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.431118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:36.431559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.432932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.441216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a645870-4cd1-4a01-880e-ee9edde634ed None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.448949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 408/808] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:36.454285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.483224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:36.490165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.490861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.491146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.491422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.492009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.494599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.497381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:36.507199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:36.508217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 15:24:36.509054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:36.510470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 401/809] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:36.510470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.575876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d743-69cc-4030-8aa9-94b9492405d2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.614977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.615953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.616732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.617626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.620187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced9c57-9c6d-4f01-98bb-ff8f59773137 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.629255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:36.635304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:36.636215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:36.637042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:36.644663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 409/810] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:36.644663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.664722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.665383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.665667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.665992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.674296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.675900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:36.689195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:36.690233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:36.691116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 402/811] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:36.691646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.759643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:36.765192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dfc70-1a5c-4010-b9a0-ffb5ff290321 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.785953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:36.805016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d02869-7ef0-4e9b-97c3-e0ffb14046c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.809648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.810527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.811005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.811497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.812315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:36.813347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.815986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.821631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 410/812] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:36.823367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.848306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.848925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.849212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.849493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.850199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:36.850511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.852240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9aa79-1216-46b9-81b5-132d45a93a21 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:36.860434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 403/813] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:36.862096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752dadb-6e54-4e00-8d5d-839ad1f3e860 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.862384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752dadb-6e54-4e00-8d5d-839ad1f3e860 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.862661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752dadb-6e54-4e00-8d5d-839ad1f3e860 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.866082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 404/814] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:36.866082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752dadb-6e54-4e00-8d5d-839ad1f3e860 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:36.949200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725005-42f8-45fc-933a-116e01b1633a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.977322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752dadb-6e54-4e00-8d5d-839ad1f3e860 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:36.983515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:36.984129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:36.984412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:36.984726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:36.985447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:36.985798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:36.987910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 411/815] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:37.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.007895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.016008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.021519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.022163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.022448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.022724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.023356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:37.023664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.025035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a4c1-e45a-4172-84da-e4a5e5a039c5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.029917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 405/816] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:37.031037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.031321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.031592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.051729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:37.053356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:37.112828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c68d-ed62-42f9-b56f-16ead53ee38e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.126541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 406/817] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:37.127436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.155453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.156192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.156546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.156890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.168502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.170085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:37.185293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:37.186312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:37.187185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:37.187185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:37.187185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:37.187185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 412/818] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:37.187983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.247476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886ba15-a2d5-4ea1-99d1-2a20f72b2a66 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.253585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.272177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.293826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.294443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.294721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.294992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.295970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:37.296278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.298457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.306999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 407/819] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:37.307359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db2f87-c9fc-4c6a-af7a-7344894066b8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.307859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.348237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.349203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.349838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.350446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.351219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.352678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.386239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 413/820] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:24:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:37.469881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.487848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13dae5-8db8-4f9d-bfbf-69aec9b47f7d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.567844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.568475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.568756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.569026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.570195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:37.570502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.571922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.598011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 408/821] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:37.608490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.646204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d43d8b-0ed1-43d3-ac05-4b24a077ea1f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.656863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.665823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:37.686780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.687429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.688260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.688260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.688627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.690595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:37.700184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:37.701191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:37.702212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 414/822] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:37.708999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.743754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58327bbd-28f7-4b09-96fb-d51fc79157c2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.814566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.815213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.815496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.815770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.822909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76943f1-6637-4ef2-9e79-8feba0068b14 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:37.834916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.836372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:37.855706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:37.856716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:37.857636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:37.861775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 409/823] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:37.869562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:37.887147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:37.888118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:37.888595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:37.889102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:37.890121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:37.890639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:37.892195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:37.901826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 415/824] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:37.908712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.006080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.015722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.018358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332029d-3772-4992-a8fc-9c07decf7d0b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.056295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.057212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.057595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.058000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.058836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:38.059242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.062412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.080798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 410/825] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:38.085905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.124694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccf186-2797-4e48-881c-3bd17e50603d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.197902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.198765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.199221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.199644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.200523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:38.201078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.202617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.209727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecdc7e-e11d-4f7f-9fc3-da32849d98c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.219554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 416/826] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:38.226579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.253271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50798f74-f000-4656-b648-78af369e6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.253965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798f74-f000-4656-b648-78af369e6756 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.254253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798f74-f000-4656-b648-78af369e6756 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.254530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798f74-f000-4656-b648-78af369e6756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.280133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50798f74-f000-4656-b648-78af369e6756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.283953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50798f74-f000-4656-b648-78af369e6756 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-50798f74-f000-4656-b648-78af369e6756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:38.312781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:38.313793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:38.315238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:38.315238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:38.315238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:38.323023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 411/827] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:38.323934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50798f74-f000-4656-b648-78af369e6756 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.346966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.419430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53563a-77f9-4a66-b108-b41618533930 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.434359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.463922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.465123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.465942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.466196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.466887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.468259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc6c5-196d-4e25-acdc-119a06d9074b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.479909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50798f74-f000-4656-b648-78af369e6756 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.497743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 417/828] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:24:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:38.497743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37a7d-d179-4d60-a833-bf2b626a439f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.497743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37a7d-d179-4d60-a833-bf2b626a439f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.497743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37a7d-d179-4d60-a833-bf2b626a439f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.497743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 418/829] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:38.512956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.513269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.513547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.545960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:38.547218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:38.568619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.569254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.569833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.570115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.570783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:38.571102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.572507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.601806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 412/830] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:38.609533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.711014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.716339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 419/831] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 12 15:24:38.761206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ed7c6-9752-480f-afac-3824d2d61850 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:38.797443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:38.798452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:38.799512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:38.800001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:38.807228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:38.809003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:38.826810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:38.827715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:38.828596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:38.830998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 413/832] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:38.836872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:38.958880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:38.967084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a22bf-c5ed-468e-817f-f203b35173bb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.012148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.012767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.013104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.013433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.014160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:39.014469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:39.015899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:39.023826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.029581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 420/833] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:39.035223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.131771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e8cb-b2bc-4e78-b9f9-a63d21a660bf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.151705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814ef5-17a1-4c06-9474-10d867dc66ed None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.155218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:39.167486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.168194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.168544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.168881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.181819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:39.187225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:39.188122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:24:39.188948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:39.191915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.192581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.192899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.193207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 414/834] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:39.193440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/b232365f81bc4933bec7a8a86778a494/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.194059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.209468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=b232365f81bc4933bec7a8a86778a494, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:39.211499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:39.230542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 421/835] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:39 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/b232365f81bc4933bec7a8a86778a494/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:39.318388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a83bb6-5db1-41cb-8a99-f75e3b446cf9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.365011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418290e-f319-485c-938e-601a08604800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.365011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418290e-f319-485c-938e-601a08604800 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.365011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418290e-f319-485c-938e-601a08604800 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.365011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418290e-f319-485c-938e-601a08604800 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.365826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418290e-f319-485c-938e-601a08604800 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:39.365826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418290e-f319-485c-938e-601a08604800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:39.366132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418290e-f319-485c-938e-601a08604800 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:39.376403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 415/836] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:39.382184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.392401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:39.558359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172c86-bb3e-4b89-8a38-80e715f66563 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.620188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.621745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.622543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.623291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.642174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:39.645465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:39.669667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:39.671489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:39.672996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:39.672996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:39.672996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:39.676275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 422/837] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:39.682815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418290e-f319-485c-938e-601a08604800 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.733421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:39.797566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418290e-f319-485c-938e-601a08604800 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:39.834355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:39.835365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.835915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.836395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.837238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:39.837797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:39.839366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d86ea-30e3-4848-b71e-0ebc3c6da404 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:39.852893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 416/838] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:39.858790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:39.879282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26fc8-9ec9-41b1-84ba-31017b228e78 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.879580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26fc8-9ec9-41b1-84ba-31017b228e78 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.879840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e26fc8-9ec9-41b1-84ba-31017b228e78 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.881315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 417/839] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:39.886441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029124b-2ce5-4d2e-8f7d-4c1250113845 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:39.886729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029124b-2ce5-4d2e-8f7d-4c1250113845 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:39.887005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029124b-2ce5-4d2e-8f7d-4c1250113845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:39.909794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d029124b-2ce5-4d2e-8f7d-4c1250113845 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:39.911038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d029124b-2ce5-4d2e-8f7d-4c1250113845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:39.943133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.002486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2c262-90ef-486a-9a57-850e970a216c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.015953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 418/840] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:40.047361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a63d9-bf34-46e9-8084-addc24b7ebb9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.047669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a63d9-bf34-46e9-8084-addc24b7ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.047918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a63d9-bf34-46e9-8084-addc24b7ebb9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.052406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.053174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 419/841] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:40.054764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.054938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.055207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.056042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.056572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.057041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.066331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.068119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.074491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:40.075760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:40.081741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:40.082751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 15:24:40.083683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.085976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 423/842] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:24:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:40.091565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.105473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.154124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 420/843] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:40.197065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af52a-dd42-4ea7-8691-44fb05ad728f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.214616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.233320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.234151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.234573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.234987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.235847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:40.236849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.238441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.257268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 424/844] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:40.263348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.287408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.398974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b29b3-47ba-4ecb-9485-bb86fd4a763b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.411870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e07f-47f5-46ba-935b-740cb9e9354b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.446563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.447168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.447444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.447720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.451139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.451766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.452041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.452312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.452884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.454287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:40.463766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:40.464905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.465960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 425/845] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:40.466771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.483267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.513866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.524807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.526305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.579329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd7ba-5dd8-42a4-8a8f-45375973a93e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.616691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.617318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.617599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.617901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.618478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.619846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.630731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 421/846] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:40.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:40.631836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:40.637807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 426/847] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:40.638607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.638678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.707138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.743418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae59d7-23bd-4170-9277-648b2d85d417 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.745698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.751043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc958d5b-5980-4b5e-a614-a3e80f292385 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.780028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.780646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.780927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.781205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.781847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:40.782263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.784174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.785940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.786568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.786846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.787348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.794404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 427/848] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:40.794404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.803129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.804568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:40.819844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:40.820761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:40.821640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:40.821640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:40.821640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:40.821640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 422/849] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:40.821640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.899232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59474c1-be4c-4997-96fc-e3ac1caabcb5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.931059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7952a-e61e-432c-9c00-4ed72c4414fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:40.938228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.938836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.939139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.939416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.940176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:40.940350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.941952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.951068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 428/850] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:40.951068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:40.951342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:40.969876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:40.971037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:40.971324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:40.971602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:40.972217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:40.972520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:40.973905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:40.978218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 423/851] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:40.979091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:41.000465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:41.085845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261e5a4-0732-4e63-857e-64ef4d7db587 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:41.129902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:41.130493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe7515-2d1a-4a88-b632-c4c45644a1f4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:41.131183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.131694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.132482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.134240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:41.137864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:41.140426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:41.158739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 429/852] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:41.168934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:41.183051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:41.183675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.183967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.184246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.184881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:41.185192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:41.188179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:41.191948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 424/853] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:41.275801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:41.286872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb6db7-0d6b-4adb-b442-74b008575512 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:41.326405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:41.327102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.327389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.327662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.335698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:41.337235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:41.350482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:41.351436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:41.352306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 430/854] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:41.360342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:41.415893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:41.483375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8536c4-103d-409e-bb0f-a37acfad4298 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:41.519409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:41.520023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.520299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.520573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.521264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:41.521570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:41.523043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f09123-9036-4949-b20c-48e13c82ed52 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:41.536202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 425/855] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:41.542791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:41.577739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442748ee-a4b1-4441-9a17-0019d9451392 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.578048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442748ee-a4b1-4441-9a17-0019d9451392 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.578278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442748ee-a4b1-4441-9a17-0019d9451392 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.579757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 426/856] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:41.591077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.591634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.591909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.623475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:41.625376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:41.665217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dadbd8-cd14-4c49-8519-248cc4441a97 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:41.701824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:41.702366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:41.702653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:41.702934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:41.703665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c9679f016e524268b81488141bfc8777) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:41.703979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:41.705988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:41.716816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 427/858] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:41.717844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 431/858] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:41 2022] GET /identity/v3/role_assignments?user.id=c9679f016e524268b81488141bfc8777&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:41.993578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.029408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:42.099064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317edf6-1dfd-450d-a565-7a99ebb55afb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.134679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.135347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.135632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.135907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:42.146970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:42.151815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:42.152717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:42.153579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 428/859] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:24:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:42.158986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.178734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:42.272365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb889a8-f3f3-4d88-97d2-9669396a55e3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.313690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.314596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.314883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.315156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:42.315820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:42.316129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:42.317492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:42.330916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 432/860] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:42.332245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.431695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:42.438895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d8aeb-9719-4770-be06-c63e2f6013d9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.475555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.476187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.476466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.476741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:42.487008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:42.488456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:42.503264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:42.504249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:42.505147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:42.505147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:42.505147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:42.509053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 429/861] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:42.510441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.594025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:42.621492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadf9a-cf83-48e6-806d-3c4ea48471ec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.658939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.659587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.659858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.660125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:42.660810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:42.661116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:42.662572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:42.679120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 433/862] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:42.680771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.784778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52780-7a82-4ecf-b4c4-61ce41d26fa2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.821079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.821735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.822032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.822315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:42.831515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:42.833028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:42.844483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:42.845461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:42.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 430/863] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:42.851373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:42.961845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7959ca-74b6-41a5-b8c4-98c931b6e8c2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:42.998433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:42.999043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:42.999319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:42.999589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.000315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:43.000616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.002646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.016860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.025752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 434/864] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:43.030777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.062102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.136974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f068c-5283-4bb2-8165-ad0981c33f77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:43.174075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:43.174760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.175057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.175344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.249276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.250805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a1ec-e6cb-411b-9cda-c0acaa57bd0f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.259721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.362685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 431/865] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:24:43.368755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.407779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e735-ad96-4f8d-8790-bc7a9e5f8690 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.408116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e735-ad96-4f8d-8790-bc7a9e5f8690 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.408305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e735-ad96-4f8d-8790-bc7a9e5f8690 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.409925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 432/866] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:43.414974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11481f-9c4d-4173-a70f-35c5add1b1a2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.415100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11481f-9c4d-4173-a70f-35c5add1b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.415292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11481f-9c4d-4173-a70f-35c5add1b1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.436538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2d11481f-9c4d-4173-a70f-35c5add1b1a2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:43.437931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2d11481f-9c4d-4173-a70f-35c5add1b1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:43.485125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2740d-2ace-4684-ac9f-83bdb1eb0672 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:43.514393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 433/867] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:43.515642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25db98-820e-4053-922c-bbe67877978f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.515936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25db98-820e-4053-922c-bbe67877978f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.516215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25db98-820e-4053-922c-bbe67877978f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.517826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 434/868] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:43.523258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.523543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.523816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.527143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:43.528058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.528602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.529070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.540123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.541956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.542637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:43.544026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:43.559150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:43.560120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:43.560993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:43.560993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:43.560993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:43.565443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 435/869] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:43.566617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.579441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.620306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.638060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 435/870] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:43.674559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f6655-3d04-415a-a735-c336db2255b2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:43.708883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:43.709486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.709799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.710075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.710711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:43.711012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.712375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.729273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 436/871] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:43.730709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.809696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.840296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeba3d-8d88-4cd5-8570-a09d19d1f407 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:43.880306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:43.881304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.881913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.882393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.889966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.891730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:43.904984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:43.905912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:43.906839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:43.909849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 436/872] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:43.915170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:43.931842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bde7f-f7b8-4230-9d80-ba35bc1f1aff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:43.942249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.946006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:43.972102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:43.972818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:43.973095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:43.973368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:43.974051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:43.975600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:43.997781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 437/873] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:43 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:24:43.997781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:44.027196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278ab77-0745-41cd-b668-90f1de400513 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:44.083499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:44.084241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.084532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.084810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.085969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:44.087164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ee79-42f4-4883-a406-6d33b1e10fb9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:44.117768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 437/874] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:43 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:24:44.138231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139179fb-be9c-4d79-924b-786f9785ad6a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.138569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139179fb-be9c-4d79-924b-786f9785ad6a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.138801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139179fb-be9c-4d79-924b-786f9785ad6a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.140552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 438/875] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:44.147187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.147600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.147733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.148629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e37a8-91a9-4e99-aaca-6f1f143bc64b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:44.167629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:44.168885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:44.204927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:44.205486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.205804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.206082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.207472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:44.207472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:44.209135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:44.223102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 438/876] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:44.230920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:44.262608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:44.284758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 439/877] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:44.357937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e085-8a8a-46a7-941a-e7d499a152b9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:44.395377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:44.396039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.396325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.396601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.397328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=b232365f81bc4933bec7a8a86778a494) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:44.397637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:44.399107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:44.405838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 439/878] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:44 2022] GET /identity/v3/role_assignments?user.id=b232365f81bc4933bec7a8a86778a494&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:44.555287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:44.571848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:44.666358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342743-7328-4cb2-81a3-8a0b348e825c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:44.704685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:44.705366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.705652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.705960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.706551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:44.707930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:44.717073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:44.718260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:44.719166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 440/879] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:44.724419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:44.815083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:44.827475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305be669-bdc6-48ca-9f6f-0b22db4c5e22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:44.863849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:44.864468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:44.864753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:44.865033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:44.865742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:44.866064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:44.867496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f17efe-a771-48f4-8873-b0ec4689a0ba None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:44.872435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 440/880] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:44.878759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:44.960213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:44.985880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c68c-0d79-4a13-8a69-d29b1b45f56d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:45.024588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:45.025195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:45.025473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:45.025768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:45.026404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:45.026705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:45.028075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d239-156d-4ab6-a372-a92d4108b6c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:45.033718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 441/881] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:46.292583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c8c5e-5d50-44c0-a0b9-7449b5aa4280 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.293141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c8c5e-5d50-44c0-a0b9-7449b5aa4280 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.293141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c8c5e-5d50-44c0-a0b9-7449b5aa4280 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.294798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 441/882] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:46.300118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2dca8-77ad-4707-a05b-53cc9d23faef None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.300552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2dca8-77ad-4707-a05b-53cc9d23faef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.300706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2dca8-77ad-4707-a05b-53cc9d23faef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.318920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7ef2dca8-77ad-4707-a05b-53cc9d23faef None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:46.320372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7ef2dca8-77ad-4707-a05b-53cc9d23faef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:46.401881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 442/883] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:46.462902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81a6a5-8a04-4a93-82fd-27243c01488b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.463190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81a6a5-8a04-4a93-82fd-27243c01488b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.463467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81a6a5-8a04-4a93-82fd-27243c01488b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.465047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 442/884] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:46.470169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e44d-7933-4eef-8e07-9110dd4afce5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.471015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e44d-7933-4eef-8e07-9110dd4afce5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.471298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e44d-7933-4eef-8e07-9110dd4afce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.489265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4672e44d-7933-4eef-8e07-9110dd4afce5 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:46.490484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4672e44d-7933-4eef-8e07-9110dd4afce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:46.554710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:46.559722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 443/885] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:46.674569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81a6a5-8a04-4a93-82fd-27243c01488b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:46.723532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211960-bbdc-4b10-981d-2c04ebae8dfa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.723934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211960-bbdc-4b10-981d-2c04ebae8dfa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.724291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211960-bbdc-4b10-981d-2c04ebae8dfa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.725938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 444/886] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:46.731097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.731469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.731755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.750460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:46.752503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:46.788132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81a6a5-8a04-4a93-82fd-27243c01488b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:46.823375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 445/887] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:46.828244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:46.829123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.829645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.830759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.831560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:46.833120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:46.840025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:46.846948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:46.847908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:46.853723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 443/888] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:46.857556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:46.880558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:46.953404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393c01e-e1a9-408c-87c1-e0e69e1da2a6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:46.982719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:46.984340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d1f1-096e-4bbc-8514-efda6e5148b3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:46.990187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:46.990826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:46.991110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:46.991389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:46.991983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:46.993360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:47.003200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:47.004194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:47.005079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:47.009202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 446/889] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Sep 12 15:24:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:47.010083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.045973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.046583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.046866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.047139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.047784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.048085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.050009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.060331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 444/890] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.061294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.129764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c51b-4cb3-4557-b1e4-fea34c933674 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.150280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.169096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.169732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.170024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.170302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.170960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.171271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.172486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57116c0-cca1-475f-a271-5c56c5a98ae7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.172660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.178258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 447/891] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.178818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.218545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.219138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.219422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.219735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.220759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.220759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.222197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.227971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 445/892] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.228739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.247068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.290310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8441f-3107-417c-bb05-831171559d48 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.324550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.325175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.325456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.325757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.331030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452b2b2-b061-4b07-abb6-3fb0591f7863 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.335719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:47.340980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:47.341980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 448/893] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:47.342821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.373186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.373830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.374116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.374401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.375032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.375342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.376752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28ee4-f340-4723-8d4c-84fbd28ff6a4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.381520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 446/894] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.382629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612934f-9737-4408-9801-8964b38e00ba None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.382919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612934f-9737-4408-9801-8964b38e00ba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.383195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612934f-9737-4408-9801-8964b38e00ba None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.384545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 447/895] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:47.386792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612934f-9737-4408-9801-8964b38e00ba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.398272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.449008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7139c6c-76ca-41e0-b047-e92e14e7282f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.465259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.482463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.483075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.483359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.483634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.484330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] RBAC: Adding query filter params (interface=public, service_id=5b6c85c044d14ba7867f49696ffa6781, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.484642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.486581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.491466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 449/896] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/endpoints?service_id=5b6c85c044d14ba7867f49696ffa6781&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.492321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.495000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612934f-9737-4408-9801-8964b38e00ba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.529186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.529822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.530108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.530380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.531364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.531364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.532685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b866ba-7178-4413-9c7d-799cd1439a4b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.540922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 448/897] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.542069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.542355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.542633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.560197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:47.561394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:47.595176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c67d-5df8-4efb-bc01-c5016924177e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.621950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.637800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 449/898] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:47.637800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.638038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.638038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.638038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.638038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.638326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] RBAC: Adding query filter params (interface=public, service_id=5fa33798fad944f3a294bd838f199685, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.638326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.638326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.643054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 450/899] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/endpoints?service_id=5fa33798fad944f3a294bd838f199685&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:47.643054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.693111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.743528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545efb0-40dd-4aed-8ba1-1dc848df8ad2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.746662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f14608-4ed1-4fb2-99b8-8e9eee448efb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.779530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.779917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.780138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.780402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.780581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.780685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.780848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.781097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.781272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.782670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.797872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 450/900] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:24:47.797872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.801400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.803073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:47.819234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:47.820818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:47.821993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:47.821993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:47.821993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:47.827472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 451/901] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:47.828456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.874190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.927602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303b913-3988-4e18-a024-757bad11ed06 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.940087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:47.944963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f242d3c-fe15-437b-9788-2c3d7dd884fa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:47.972752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.973379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.973657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.973974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.974587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.975984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:47.982681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 451/902] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:24:47.983813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:47.990882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:47.991535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:47.992505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:47.992793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:47.993482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:47.994088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:47.996533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.012138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 452/903] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:48.018377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:48.121202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad17ce-29b2-49c8-bf00-5e7405d08bad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:48.139145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ac83-da1d-43bf-bef4-98c6b2303f49 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:48.154207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:48.164292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:48.165533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:48.166033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:48.166544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:48.167378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:48.168931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.179967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:48.180557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:48.180926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:48.181118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:48.190123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:48.190662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 452/904] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:47 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:48.191565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:48.206059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:48.207096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:48.208059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 453/905] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:48.213359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:48.336877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27997f-b2ec-4be6-8ea1-ece16ab2e67d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:48.379279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:48.379902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:48.380186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:48.380467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:48.381233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:48.381543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:48.383007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.405540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 453/906] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:48.411689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:48.432837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:48.520816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4297e-0393-4c97-9fa8-5c4534e1a1fe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:48.557485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:48.558178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:48.558461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:48.558732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:48.632647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:48.634152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.653603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:48.741745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 454/907] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:24:48.747476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:48.795922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:48.858296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285591-9b10-4cde-b6b2-a8c0e58b67e4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:48.894840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:48.895479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:48.895765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:48.896044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:48.906758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:48.908255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:48.923061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:48.924026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:48.924908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:48.924908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:48.924908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:48.924908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 454/908] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:48.930170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:49.036388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0439aa7-45db-44c8-bf35-bea13d61e065 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:49.054758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:49.074644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:49.075607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:49.076088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:49.076561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:49.077396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:49.077935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:49.079512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:49.096478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 455/909] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:49.101959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:49.203768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:49.210642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e834-d00e-4410-a240-c338c424317f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:49.252744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:49.253374 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:49.253656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:49.253950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:49.260936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:49.262411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:49.273799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:49.274674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:49.275658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 455/910] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:49.281030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:49.385915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf0280-3780-46d6-b880-b0fd8564d9e0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:49.420848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:49.421489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:49.421746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:49.422031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:49.422727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:49.423039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:49.425010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6fea0-04e4-4892-b686-ee99c2720bf4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:49.437652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 456/911] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:49.444154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:49.550446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab4fd1-9d1e-46cc-a5a3-731ae43f1122 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:49.584548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:49.585157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:49.585438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:49.585724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:49.586553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] RBAC: Adding query filter params (role.id=3b0fe195021a4ab6aa2049dfb765c910, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=b232365f81bc4933bec7a8a86778a494) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:49.586994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:49.588876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7a48b-ab06-4c03-b958-c238249683a1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:49.596884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 456/912] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:49 2022] GET /identity/v3/role_assignments?user.id=b232365f81bc4933bec7a8a86778a494&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=3b0fe195021a4ab6aa2049dfb765c910 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:50.129393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277bf7-b570-46d2-a9fd-54fc8efd8c37 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.129797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277bf7-b570-46d2-a9fd-54fc8efd8c37 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.130005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277bf7-b570-46d2-a9fd-54fc8efd8c37 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.131538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 457/913] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:50.136831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc63fb-19df-403e-9517-3047d9c48b1a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.137122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc63fb-19df-403e-9517-3047d9c48b1a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.137398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc63fb-19df-403e-9517-3047d9c48b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.156298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-02dc63fb-19df-403e-9517-3047d9c48b1a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:50.157492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-02dc63fb-19df-403e-9517-3047d9c48b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:50.171383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:50.232251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 457/914] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:50.311254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575e59-a23f-48a1-9cbf-ddd0baf7782c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.311855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575e59-a23f-48a1-9cbf-ddd0baf7782c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.312364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575e59-a23f-48a1-9cbf-ddd0baf7782c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.314267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 458/915] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:50.319870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438d18c-4423-4750-b30f-220fcdbb86f0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.320159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438d18c-4423-4750-b30f-220fcdbb86f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.320435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438d18c-4423-4750-b30f-220fcdbb86f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.342733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1438d18c-4423-4750-b30f-220fcdbb86f0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:50.343939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1438d18c-4423-4750-b30f-220fcdbb86f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:50.425847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 458/916] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:50.512391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575e59-a23f-48a1-9cbf-ddd0baf7782c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:50.544030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e03cd-70cc-4037-ab88-bb1dfa85e944 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.545093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e03cd-70cc-4037-ab88-bb1dfa85e944 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.545916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e03cd-70cc-4037-ab88-bb1dfa85e944 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.547989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 459/917] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:50.553655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.554208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.554701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.556539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:50.578063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:50.579841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:50.623399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575e59-a23f-48a1-9cbf-ddd0baf7782c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:50.651711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 460/918] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:50.665672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:50.666518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.666946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.667380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.668124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:50.669764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:50.679555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:50.680752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 15:24:50.681821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.684007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 459/919] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:50.690014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:50.698769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:50.716326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:50.805680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295ca9-ae76-4782-a826-9560fa3d11c1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:50.807335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:50.819752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f066135-689d-484f-9f3a-9c4f4c7b5fd2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:50.852884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:50.853944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.854453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.854947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.855737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:50.856775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:50.857383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:50.857668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:50.857993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:50.858358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:50.858623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:50.858929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:50.860590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:50.869350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:50.870323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 15:24:50.871228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:50.871965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 461/920] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:50.873481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:50.873661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 460/921] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Sep 12 15:24:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:50.882312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:50.971066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:50.989450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c87c8d-381a-462f-9879-337b759ce238 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.002065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebe442-2f92-4028-9d2c-e03a7d384c1f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.024964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.025827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.026106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.026374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.026998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.027303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.028667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.034341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 461/922] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:51.035223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.042664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.043538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.044016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.044483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.045296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.045903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.047479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.053112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 462/923] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Sep 12 15:24:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:51.054168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.072789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.142679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a71dce-e773-440f-b6b1-998351daa812 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.158748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a402d-5fd6-4f58-ba6b-187ab3be1c73 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.175433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.176070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.176352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.176652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.177209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.178604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:51.187257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:51.188240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:51.189639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 462/924] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:51.191104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.199802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.200533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.200819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.201095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.201741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.202050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.203544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.206495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.217051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 463/925] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:51.217051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.292838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.300786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f639-dfab-4172-853a-1a63e97fb2aa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.321776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab255a1-aa57-4bb7-bc9d-af6f176dd97e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.339008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.339575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.339857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.340339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.340789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.342200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.358983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.359640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.359920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.360188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.360841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.361314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.363207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.377226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 463/926] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:24:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:51.377439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 464/927] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:51.377532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.380433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.462449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.518331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310c84a-3803-4962-b0d8-905b7788808f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.545270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e7771-eb28-487e-9ef1-7c7bbc27939a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.571048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.571690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.571941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.572215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.573070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.573380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.574784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.580160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 465/928] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:51.585831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.598861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.599476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.599759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.600031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.600704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.602126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.627974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 464/929] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:24:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:51.705560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616064c-ccad-4bc4-aa5b-47da50db4108 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.741410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.757990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.758612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.758894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.759165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.759875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] RBAC: Adding query filter params (interface=public, service_id=453311c6328e4c7284e924d17d7e3cb3, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:51.760187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.761593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb80b-0144-4a63-82f1-fa88b4d2b132 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.767984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 466/930] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/endpoints?service_id=453311c6328e4c7284e924d17d7e3cb3&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:51.774001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:51.790093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:51.806542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ed2-5068-45ac-b8f1-5d1528a52ee4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.806926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ed2-5068-45ac-b8f1-5d1528a52ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.807214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1ed2-5068-45ac-b8f1-5d1528a52ee4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.809751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 467/931] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:51.815383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.815702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.815948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.838519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:51.839713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:51.908726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba697-3a2f-4dd5-a8ef-51a88c68634a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:51.924332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 468/932] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:51.948154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:51.948792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:51.949071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:51.949346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:51.949983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:51.951356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:51.957922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 465/933] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:51 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:24:52.200409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:52.306014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26b10c-755a-44cb-bd55-086fb4f62dcc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:52.324719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:52.343389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:52.344268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:52.344555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:52.344834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:52.356064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:52.361041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:52.362011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:52.362874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 469/934] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:24:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:52.370104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:52.443739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:52.483836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb429ed-f530-41bb-b40e-9c1b36f63989 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:52.519044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:52.519663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:52.519951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:52.520230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:52.520900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:52.521209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:52.525713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:52.537763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 466/935] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:24:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:52.543377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:52.651070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0fc9-8d66-487d-87fc-abfef843841b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:52.687509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:52.688150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:52.688435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:52.688709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:52.699947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:52.701475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:52.716758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:52.717718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:52.718623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:52.718623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:52.718623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:52.718623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 470/936] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:52.725129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:52.833353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b6be-4036-498c-a23f-da64ca9a4ff8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:52.861042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:52.870506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:52.871409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:52.871897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:52.872375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:52.873270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:52.873826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:52.875536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:52.888513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 467/937] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:52.895637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:52.931855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:53.006565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b300a6-5d66-4f40-9940-dddf716fa338 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.040797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.041424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.041721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.041999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.050137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.051683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:53.063131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:53.064622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:53.065555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 471/938] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:53.072636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.181866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ce7c5-c93b-4adb-b937-318aecd829aa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.218236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.218857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.219143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.219424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.220209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:53.220527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.221990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.240668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 468/939] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:53.246505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.342912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:53.354156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4939b-e3b5-4ee5-a673-cb0405913922 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.390861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.391498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.391785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.392093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.403391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.404902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:53.420205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:53.421147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:53.422809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:53.422809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:53.422809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:53.422809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 472/940] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:53.427986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.478536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:53.538733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7405d8-2ac0-4870-85b9-91f093f43dc9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.574152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.574754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.575033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.575308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.575947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:53.576251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.577717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee700a-496a-4920-a59d-1e7e5ae56714 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.590760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 469/941] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:53.596245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.705587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39456685-e02b-4bd2-8461-5a18c45d1116 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.726640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc81ac-819d-49a3-bda1-6c2801c9a958 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.727152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc81ac-819d-49a3-bda1-6c2801c9a958 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.727229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc81ac-819d-49a3-bda1-6c2801c9a958 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.728841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 470/942] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:53.734128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.734666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.735145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.740464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.741160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.741447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.741741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.748455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.749933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.756886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:53.758618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:53.763512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:53.764389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:53.765278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 473/943] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:53.770239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.782286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:53.858162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 471/944] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:53.885628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a064a21-c57e-427b-a8dd-00adb91f595e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:53.923920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:53.924532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:53.924818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:53.925097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:53.925890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:53.926201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:53.927634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa6ee-f2de-4436-b51e-a9f548176571 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:53.941542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 474/945] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:53.948110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:53.992136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.063052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef497f78-3f4d-4bbc-b956-0f19edf92e8b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.068533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938d75-09eb-4cb6-8bca-71ff53b7c238 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.068820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938d75-09eb-4cb6-8bca-71ff53b7c238 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.069095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938d75-09eb-4cb6-8bca-71ff53b7c238 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.070974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 475/946] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:54.075522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11211f47-e4e3-4444-99bc-44c489447c2f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.075911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11211f47-e4e3-4444-99bc-44c489447c2f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.076284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11211f47-e4e3-4444-99bc-44c489447c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.098078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11211f47-e4e3-4444-99bc-44c489447c2f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:54.100132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11211f47-e4e3-4444-99bc-44c489447c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:54.105278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.106173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.106613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.107041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/b232365f81bc4933bec7a8a86778a494/roles/3b0fe195021a4ab6aa2049dfb765c910` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.109758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=b232365f81bc4933bec7a8a86778a494, role_id=3b0fe195021a4ab6aa2049dfb765c910)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.112299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.130710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 472/947] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:24:53 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/b232365f81bc4933bec7a8a86778a494/roles/3b0fe195021a4ab6aa2049dfb765c910 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:24:54.165506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.171135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.184283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 476/948] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:54.293348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.301389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0859f-6cd2-41f0-a86d-c38271f3b1a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.339254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.339917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.340213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.340486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.341135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.342623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.345816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING keystone.server.flask.common [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 15:24:54.368527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 15:24:54.368527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 15:24:54.368527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [parameters: {'id': 'd9104f17154f4693b3a73f5f45372966', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 15:24:54.368527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 15:24:54.385316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0354-25e8-4f9b-b841-c9e321f0748e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.385607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0354-25e8-4f9b-b841-c9e321f0748e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.386005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0354-25e8-4f9b-b841-c9e321f0748e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9104f17154f4693b3a73f5f45372966.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9104f17154f4693b3a73f5f45372966. Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:24:54.386110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 12 15:24:54.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 12 15:24:54.388375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 15:24:54.390603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 15:24:54.394090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:24:54.396201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'id': 'd9104f17154f4693b3a73f5f45372966', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.398387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:54.400298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9104f17154f4693b3a73f5f45372966. Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.405355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 473/950] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Sep 12 15:24:54 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:54.406686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 477/950] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:54.406837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.414666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.414666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.414666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.435329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:54.436820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:54.506118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb649f-3e2a-4a8c-a7e8-fc8be400b2c3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.510335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 478/951] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:54.514619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.554212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.555136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.555568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.556832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.567238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.569979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:54.584754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:54.585857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:54.586900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:54.588596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 474/952] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:54.594538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.612837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.617728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fc2f7-6e5b-4b59-867e-3e18c9e589f7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.641022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.655128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.655769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.656036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.656314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.656897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.659056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:54.668458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:54.669680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:24:54.670910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:54.674495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 479/953] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:54.675430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.717115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182a32-5c45-4594-8af2-49536bcc9c19 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.753135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.754526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.754994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.755463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.756548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:54.757040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.758700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.779605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf818ad8-179c-4927-a07c-4bb3cb398bfe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.786308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 475/954] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:54.797743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.820563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.821099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.821377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.821650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.822319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:54.822624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.824031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:54.834681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 480/955] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:54.842734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:54.904743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.907859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:54.950124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0473c-e1e6-4d32-89b3-de85051aa949 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.966039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72f34-0c0d-418b-b446-53dcab964fd6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:54.988412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:54.989297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:54.989659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:54.990502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:54.990824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:54.992203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:55.004391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:55.005476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:55.007159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 476/956] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:55.010177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:55.011330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:55.012111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:55.012393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:55.013097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:55.013774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:55.014073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:55.015452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:55.021340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 481/957] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:24:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:55.151124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ddc7a-8a8d-4743-bb6f-c3816c34562a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:55.190791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:55.191666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:55.192092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:55.192672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:55.193455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:55.193986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:55.195751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:55.198146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:55.202696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 477/958] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:55.208790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:55.324639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40031e8b-108d-4e43-953a-7be6c5487685 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:55.338793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:55.360138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:55.361574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:55.362081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:55.362544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:55.363357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:55.363850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:55.365400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d8384-e680-408f-9029-92a21fd40112 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:55.371406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 482/959] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:55.378308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:55.491181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31382b83-ec81-4c21-990e-1dfa29517e6c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:55.526295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:55.526917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:55.527201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:55.527533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:55.528259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:55.529637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9aab9-372e-4ce1-9dd7-32ac02bcd4d7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:55.551371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 478/960] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:24:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:56.420656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2be27-796e-42db-9cd7-8788279f1994 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:56.421169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2be27-796e-42db-9cd7-8788279f1994 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:56.421275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2be27-796e-42db-9cd7-8788279f1994 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:56.422853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 483/961] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:56.428720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:56.429231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:56.429328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:56.449974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:56.451507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:56.504377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:56.528345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 479/962] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:56.804315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2be27-796e-42db-9cd7-8788279f1994 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:56.910575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2be27-796e-42db-9cd7-8788279f1994 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:56.948993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:56.950047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:56.950580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:56.951046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:56.962625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b98d21c9-407a-45a2-b225-66c504189a9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:24:56.967657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:56.968565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:24:56.969380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:56.972051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 484/963] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:24:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:56.977729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.089525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176820f6-561d-4014-9e4a-2fb09bcd0e65 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.126017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.126720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.127005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.127281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.127955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:57.128260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.129639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.138689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 480/964] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:24:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:57.140032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47a22-39cb-468d-a43a-6a02a7549ed2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.140325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47a22-39cb-468d-a43a-6a02a7549ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.140605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47a22-39cb-468d-a43a-6a02a7549ed2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.142229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 485/965] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:57.147050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.147522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a567b-de2e-4725-90ac-f9a06c69c362 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.147811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a567b-de2e-4725-90ac-f9a06c69c362 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.148079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a567b-de2e-4725-90ac-f9a06c69c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.167829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1c7a567b-de2e-4725-90ac-f9a06c69c362 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:57.169068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1c7a567b-de2e-4725-90ac-f9a06c69c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:57.201314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.202646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.249986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 486/966] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:57.267729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926e7d-f661-4603-b50c-f11db30ee214 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.280926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ecfae-d51b-4800-bf69-3f2403b42d36 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.281302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ecfae-d51b-4800-bf69-3f2403b42d36 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.281651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ecfae-d51b-4800-bf69-3f2403b42d36 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.283978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 487/967] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:24:57.289041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.289335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.289616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.305558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.306229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.306512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.306788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.309124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:57.310350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:57.322628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.324800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:57.345583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:57.347499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:57.348834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:57.348834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:57.348834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:57.348834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 481/968] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:57.352500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.390123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 488/969] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:24:57.475703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.482490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0314-745f-4983-b842-27fe549ec8ee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.518593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.519529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.520010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.520464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.521293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:57.521846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.523399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.531207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.536922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 482/970] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:57.542470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.632960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.638200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3121ff-9512-40c4-b326-cc241a1588b9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.645482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902de521-238b-432c-a7ad-f1e5e472f813 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.679641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.680371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.680725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.681162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.681933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.683594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.685174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.686079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.686539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING keystone.server.flask.common [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 15:24:57.686734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.687173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.696686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.698626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.701090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.sql.core [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 15:24:57.701090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 15:24:57.701090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [parameters: {'user_id': '97d842d5b8614034a1994510d88eae56', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 15:24:57.701090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81400) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Sep 12 15:24:57.708481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:24:57.709509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application yield resource Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application next(self.gen) Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise self.value Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application session.commit() Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 15:24:57.711011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.session.flush() Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application compat.raise_( Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 15:24:57.711971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 15:24:57.712870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application conn.query(q) Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application result.read() Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 15:24:57.714803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [parameters: {'user_id': '97d842d5b8614034a1994510d88eae56', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:57.716929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 12 15:24:57.719253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 15:24:57.721471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:57.722376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:57.724485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 15:24:57.726811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:57.728137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 489/971] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Sep 12 15:24:57 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:57.728137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.728419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 483/972] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:57.728419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.737404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.829822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949b83d-3f27-4c61-9951-2889187b98e4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.832636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d335-e9fb-41a6-8d4f-36ffa8593b2c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:57.863994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.864630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.864920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.865194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.866338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:57.866943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.867224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.867494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.868131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.869678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.872659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:57.874136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-206eba60-6d88-4c82-970b-2cb16a38b3ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:24:57.886384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:57.887361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 15:24:57.888306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:57.888873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 490/973] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:24:57 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:24:57.888952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 484/974] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:57.889028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57bc2a-bbe2-4a6c-ba1d-8a1a07d34720 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:57.889298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57bc2a-bbe2-4a6c-ba1d-8a1a07d34720 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:57.889584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57bc2a-bbe2-4a6c-ba1d-8a1a07d34720 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:57.889993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.895622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 485/975] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:24:57.895622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57bc2a-bbe2-4a6c-ba1d-8a1a07d34720 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:57.922095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:57.994941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.022903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab7d95-1a8c-4fc1-8461-4bd14eef46df None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.028253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57bc2a-bbe2-4a6c-ba1d-8a1a07d34720 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.066968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.067587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.067869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.068139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.068868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:58.069997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.070149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.070244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.070483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.070812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.070930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44fb04-8f65-41a7-ad97-e74fde4bdcb6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.071624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:58.071998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.074190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.090884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 491/976] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:58.092261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.093413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.093698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.117082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:24:58.118807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 486/977] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Sep 12 15:24:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:58.118930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:24:58.119569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.219539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 492/978] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:24:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:24:58.252177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb3efb-8cb2-487b-ac46-ceb8d6e7abbd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.288316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.297652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.298844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.299132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.299414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.378539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.380099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.492743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 487/979] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:24:58.498456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.501960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.526785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.567183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.607747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f0e93-a15c-4bb0-999d-cd7d4758814a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.640648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b241de-b6e8-4a18-b83d-52e5fa570b30 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.646141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.646850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.647202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.647546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.657944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.659473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:24:58.674570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:58.675712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:24:58.681465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:24:58.681465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:24:58.681465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:58.681465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 493/980] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:24:58.681776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.683287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568418c-3588-4099-a35e-146bda92276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.684000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568418c-3588-4099-a35e-146bda92276b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.684361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568418c-3588-4099-a35e-146bda92276b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.684704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568418c-3588-4099-a35e-146bda92276b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.685370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568418c-3588-4099-a35e-146bda92276b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.686860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568418c-3588-4099-a35e-146bda92276b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4568418c-3588-4099-a35e-146bda92276b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:58.697087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:58.698408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:24:58.699358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 488/981] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:58.704426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568418c-3588-4099-a35e-146bda92276b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.745729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.798186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878496f-8db0-47c6-9d23-6ec9c7f75a71 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.810881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:58.819036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568418c-3588-4099-a35e-146bda92276b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.833792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.834399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.834678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.834945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.835580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:58.835885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.837302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.850474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 494/982] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:58.854636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:58.855694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:58.855977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:58.856147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.856260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:58.856865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:58.857252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:58.858714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:58.864242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 489/983] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:58.869695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:58.971059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f16f67-3db2-4011-8cc7-081e6878fe52 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.980826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0247da2-21d5-4840-96d5-79f5ecf1cefe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:58.988730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:59.008985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.009548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.009867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.010145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.017721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.019162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.019285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.019774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.020063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.020343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.020972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:59.021293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.022670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.027662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 490/984] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:24:59.031288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:24:59.032170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:24:59.033799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 495/985] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:24:59.034910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:59.039214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:59.061165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:59.147388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6e42c-edc8-4326-80a8-da354383fed8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:59.147618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bc124-5137-4006-b756-d6fa8a9edeb0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:59.183736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.184283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.184572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.184852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.185543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:59.185959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.187330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.187455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.188036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.188317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.188590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.189342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] RBAC: Adding query filter params (interface=admin, service_id=453311c6328e4c7284e924d17d7e3cb3, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:59.189650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.191013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.202241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 491/986] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Sep 12 15:24:59 2022] GET /identity/v3/endpoints?service_id=453311c6328e4c7284e924d17d7e3cb3&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:24:59.202399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 496/987] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:24:59.203085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:59.206007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:24:59.242069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:59.293577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:24:59.309219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32ef8-8366-4fc9-90af-54a1b86d4b14 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:59.312103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330b4ae-e57b-43db-8c41-a5ebecf10838 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:24:59.343940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.344551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.344836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.345638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.346162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:24:59.346275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.346760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:24:59.347040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:24:59.347311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:24:59.347634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fd08f-ce43-4685-8270-82b29c265120 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.348025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] RBAC: Adding query filter params (role.id=3b0fe195021a4ab6aa2049dfb765c910, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=b232365f81bc4933bec7a8a86778a494) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:24:59.348389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:24:59.349876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367183-2a26-4701-ba63-aa534590fed9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:24:59.366092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 492/988] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:24:59 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:24:59.366319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 497/989] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:24:59 2022] GET /identity/v3/role_assignments?user.id=b232365f81bc4933bec7a8a86778a494&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=3b0fe195021a4ab6aa2049dfb765c910 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:00.068565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ace-5d44-457a-9610-aa4efa9a873f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:00.069097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ace-5d44-457a-9610-aa4efa9a873f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:00.069097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ace-5d44-457a-9610-aa4efa9a873f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:00.070800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 493/990] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:00.075959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:00.076401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:00.076580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:00.094913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:00.096654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:00.166403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 498/991] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:00.429578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9ace-5d44-457a-9610-aa4efa9a873f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:00.533448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9ace-5d44-457a-9610-aa4efa9a873f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:00.567840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:00.568471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:00.568757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:00.569034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:00.569892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:00.571258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:00.580528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:00.581608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:00.583788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 494/992] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:25:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:00.588063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:00.617901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:00.687752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f1b9f-51c0-439c-9fc2-dfe2963236ae None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:00.721342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:00.722102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:00.722489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:00.722861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:00.723584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:00.724102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:00.725468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:00.730573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 499/993] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:25:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:00.736489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:00.753541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:00.843325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ca35c-d841-464b-8b7a-3982b948c911 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:00.878239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:00.878966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:00.879352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:00.879719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:00.880487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:00.880908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:00.882406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:00.891899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 495/994] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:25:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:00.893144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:00.994565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:01.002350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3150-1889-4849-be9e-975a1405bba3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:01.036204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:01.036898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.037248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.037591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.038379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] RBAC: Adding query filter params (interface=public, service_id=fc0b952be41c4c00a9222bc9755ae7b5, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:01.038757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:01.040188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6727d-a5bc-463f-99bf-1b20e444e40b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:01.045428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 500/995] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:25:00 2022] GET /identity/v3/endpoints?service_id=fc0b952be41c4c00a9222bc9755ae7b5&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:01.050915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:01.131111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:01.154997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f5be-a892-466c-aaab-4c8808f0eacd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:01.190796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:01.191410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.191693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.191966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.192566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:01.193963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1277f3-8a69-46c6-9356-f5ebfd38c1f9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:01.205788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 496/996] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:25:01 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:25:01.543804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f53d2-528f-4f2e-a81e-5f07c40d1c65 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.544326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f53d2-528f-4f2e-a81e-5f07c40d1c65 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.544406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f53d2-528f-4f2e-a81e-5f07c40d1c65 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.545895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 501/997] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:01.551460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.551750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.552212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.553436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dc620-1839-4ad5-84e6-831981c1a562 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.553743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dc620-1839-4ad5-84e6-831981c1a562 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.554024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dc620-1839-4ad5-84e6-831981c1a562 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.555422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 502/998] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:25:01.560006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:01.560294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:01.560568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:01.572067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:01.573274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:01.578901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:01.580108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:01.644147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 497/999] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:01.649640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 503/1000] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:25:01.908050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:01.914781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:01.979095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:02.011601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28948291-43b0-4527-825a-d9330b5b5957 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.015945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:02.033686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefbd24-696a-4c58-9e67-7a9617882f6b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.049440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.050357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.050731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.050982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.051492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.052888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:02.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:02.063051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:02.063960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 498/1001] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Sep 12 15:25:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:02.069272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:02.078005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.078659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.078940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.080338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.080941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.082730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:02.092674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:02.093692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:02.094559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 504/1002] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Sep 12 15:25:01 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:02.099478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:02.176577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1826e-05b4-457d-9439-ed07665dcd3a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.201845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde09a93-6828-4fa6-bc60-a029abf52d8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.211325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.211924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.212214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.212592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.213210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:02.213515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.214917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.216819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:02.223155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 499/1003] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:25:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:02.228795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:02.245343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.246101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.246407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.246730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.247357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:02.247662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.249008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.258197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 505/1004] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:02.259171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:02.277958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:02.339839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bca6e-d8fd-4165-8699-e2019ca7b354 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.368329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc4c3-fba8-4468-aaed-0db1280ea5c5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.381752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.381752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.381752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.381752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.382647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:02.382647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.382778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.387998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 500/1005] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Sep 12 15:25:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:02.393511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:02.405480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.406118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.406398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.406671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.407299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:02.408144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.409519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c06bf-e981-41bb-b906-3ec7401c1fdf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.421750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 506/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:02.467122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:02.510467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4a83-61c2-474d-b9bb-e6a1ce06d2eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:02.547022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:02.547648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:02.547929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:02.548261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:02.548942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:02.551736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c608c-5b90-4dc6-8a29-6bc8ce07eb19 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:02.573860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 501/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:25:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:25:03.358506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba432a-b91e-4381-9bd0-fa002bf1810a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:03.359061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba432a-b91e-4381-9bd0-fa002bf1810a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:03.359061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba432a-b91e-4381-9bd0-fa002bf1810a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:03.360704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 507/1008] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:03.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:03.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:03.368806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:03.386861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:03.388060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:03.463602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 502/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:03.728800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba432a-b91e-4381-9bd0-fa002bf1810a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:03.810918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:03.833432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba432a-b91e-4381-9bd0-fa002bf1810a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:03.866286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:03.866915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:03.867197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:03.867471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:03.868052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:03.869421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:03.878765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:03.879730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 15:25:03.880580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:03.882168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 508/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:25:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:03.885679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:03.935608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:03.988555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f3cae-a92c-43da-a5d0-315fda7e767e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:04.023583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:04.024194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.024478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.024753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.025386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:04.025696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:04.027094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:04.032099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 503/1011] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:25:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:04.037595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:04.136866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127bb97-57f9-4091-a7c6-cc7c0ee5512a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:04.170968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:04.171574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.171859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.172137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.172773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:04.173595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:04.174997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da8a67-1dd4-436d-894b-20555f1bcc32 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:04.184480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 509/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Sep 12 15:25:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:04.185682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:04.294002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b28a-0e5a-4ef3-8ba5-5b8020999559 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:04.308965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:04.331338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:04.331957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.332241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.332516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.333186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:04.334579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0027-17a6-4335-86a5-15e1ba09c0ad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:04.356295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 504/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:25:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:04.571496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee275-e2c8-4db6-bc5e-02e5276db5a0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.572345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee275-e2c8-4db6-bc5e-02e5276db5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.572623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ee275-e2c8-4db6-bc5e-02e5276db5a0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.574158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 510/1014] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:04.579619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2c35-5f9a-4e6b-9e8e-76dac56197c0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.580084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2c35-5f9a-4e6b-9e8e-76dac56197c0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.580171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2c35-5f9a-4e6b-9e8e-76dac56197c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.599215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ec9c2c35-5f9a-4e6b-9e8e-76dac56197c0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:04.600762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ec9c2c35-5f9a-4e6b-9e8e-76dac56197c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:04.685752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 505/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:04 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:04.733668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275a59b-7f10-40c8-946d-250e9a56ba11 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.733996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275a59b-7f10-40c8-946d-250e9a56ba11 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.734265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275a59b-7f10-40c8-946d-250e9a56ba11 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.736208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 511/1016] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:25:04.741583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133608-4193-4991-9f79-27278029947e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:04.741920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133608-4193-4991-9f79-27278029947e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:04.742201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133608-4193-4991-9f79-27278029947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:04.760329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d5133608-4193-4991-9f79-27278029947e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:04.761524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d5133608-4193-4991-9f79-27278029947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:04.829818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 506/1017] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:04 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:04.953750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275a59b-7f10-40c8-946d-250e9a56ba11 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:04.968542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:05.067143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275a59b-7f10-40c8-946d-250e9a56ba11 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:05.100196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:05.100654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5133608-4193-4991-9f79-27278029947e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:05.100772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:05.100995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:05.101269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:05.101955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:05.103321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:05.118866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 512/1018] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:25:04 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:05.133465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:05.217408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5133608-4193-4991-9f79-27278029947e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:05.263184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:05.263898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:05.264178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:05.264452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:05.265772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:05.267377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0df8bd-90b7-4c8d-8656-97a76ab8ec05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:05.269250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 507/1019] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:05.275326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:05.348962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:05.393980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c0b79-760e-4431-9574-cb08728f434c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:05.426930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:05.427539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:05.427825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:05.428101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:05.428747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:05.430200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d0e11-7839-4241-8ca1-5b7816ecd945 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:05.458621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 513/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:25:05 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:06.657633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88446ad-487d-4d22-873d-aed6bad80b8f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:06.658218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88446ad-487d-4d22-873d-aed6bad80b8f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:06.658218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88446ad-487d-4d22-873d-aed6bad80b8f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:06.659639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 508/1021] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:06.664681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:06.665072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:06.665235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:06.683775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:06.684971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:06.753446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 514/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:07.033725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88446ad-487d-4d22-873d-aed6bad80b8f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.127197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.139181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88446ad-487d-4d22-873d-aed6bad80b8f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.174553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.175190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.175476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.175753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.186615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7a6da15b-dbdb-43d6-a401-1fbc2c08972d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:07.191792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:07.192805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:07.193650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:07.197976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 509/1023] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:07.199725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.251544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.304662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da70f31-44ed-4b49-bece-01426e2f4790 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.310325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8f958-2e95-4e4e-b0f1-4d52ce23f045 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.310610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8f958-2e95-4e4e-b0f1-4d52ce23f045 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.310884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8f958-2e95-4e4e-b0f1-4d52ce23f045 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.313662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 510/1024] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:25:07.319001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.319288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.319565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.338473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:07.339662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:07.341823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.342437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.342721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.342997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.343665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:07.343972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:07.345936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:07.354181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 515/1025] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:07.360567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.426701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 511/1026] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:25:07.460768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674b827-7c9a-4b49-8903-ac2133486bde None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.473944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.496358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.497221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.497644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.498091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.508931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:07.510608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-972e12f2-a2c8-4101-8ecd-7207f639d88d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:07.527278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:07.528454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:07.534599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:07.534599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:07.534599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:07.534599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 516/1027] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:07.537101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.623133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.641075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f974-1522-40a7-9813-70ea7c77bd4e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.641366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f974-1522-40a7-9813-70ea7c77bd4e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.641642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f974-1522-40a7-9813-70ea7c77bd4e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.643189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 517/1028] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:25:07.643783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce6387-5cfd-4b49-be9b-2121e27b150e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.650208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.650501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.650783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.669665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:07.672430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:07.681292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.681933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.682234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.682500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.683140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:07.683444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:07.684815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:07.696327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 512/1029] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:07.697128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.745007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 518/1030] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:25:07.745878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.800027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096abbd-9c29-4838-b1fe-9c8589b5f598 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.834285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.835073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.835194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.835450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.836027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:07.837396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:07.843785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:07.846764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:07.848522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:07.850206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 513/1031] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:07.853138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c29960-10c3-4b33-897d-3016dd6bbe63 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:07.853621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.858442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:07.888835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:07.889398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:07.889811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:07.889997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:07.897033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:07.898507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:25:07.910701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:07.911656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:07.912603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 519/1032] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:07.917350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:07.979110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9f342-fac3-4e20-ac8c-3c9adcf8cbc0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.018093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e78e2-a48b-44b6-8988-8735f3f6e17f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.020018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.020633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.020916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.021194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.021865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:08.022173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.023549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.028640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 514/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:08.029243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.052922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.053532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.053860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.054133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.054860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:08.055169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.056630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.073663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 520/1034] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:25:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:08.074477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.090063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.100662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.133504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b840a-2ca1-4b89-ba72-bbc83fe05f84 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.167324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.167883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.168161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.168431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.169580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.171037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.172587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 515/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:08.173363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.178325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdbf8f-4fd7-464e-a862-e67e3d3505a5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.212604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.213215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.213495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.213797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.214423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:08.214729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.216094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.221566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 521/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:08.222197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.276322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa5d8-7ccd-4487-bad5-0802934a2289 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.295782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.310884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.311491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.311776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.312120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.315960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.319213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7496-32d5-4563-badc-595df2e1726c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.352862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.353455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.353761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.354043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.354690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.356098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.380356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 522/1037] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:25:08 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:08.381184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.383973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.385449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.496870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8356936-bc75-4daf-be7e-05cb8d822030 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.500070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 516/1038] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:08.506611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.539612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.544608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.549321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.549983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.550312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.550765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.551292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] RBAC: Adding query filter params (interface=public, service_id=3f11fd73318a4473823500ede1e3dd3a, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:08.551786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.554264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.560803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 523/1039] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/endpoints?service_id=3f11fd73318a4473823500ede1e3dd3a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:08.566521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.626358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443ee1-ad3f-421f-933f-89a96e88a3b6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.664576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.665324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.665608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.665914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.677350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc0bc7-2072-4843-9db7-0ac344968cb5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.677836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.679091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:08.696063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:08.697463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:08.698399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:08.698399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:08.698399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:08.698399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 517/1040] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:08.705774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.713380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.714026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.714313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.714588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.715195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.716562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.724395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 524/1041] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:25:08.802050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.817263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ca7e5-b3f5-46ce-bbcf-905a513d54b9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:08.854306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:08.854934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:08.855212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:08.855487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:08.856135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:08.856442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:08.857881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:08.875595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 518/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:08.876943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:08.941200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:08.985773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd5713-79ac-4999-845b-292806ada970 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:09.021364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:09.023495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:09.023495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:09.023495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:09.030254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:09.031693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:25:09.044054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:09.044929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:09.045846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 525/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:09.051016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:09.164641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1ccd-ef6f-40e5-96ea-38c3b85b8356 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:09.200478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:09.201097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:09.201389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:09.201667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:09.204037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:09.204037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:09.204307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fc566-3e03-41f5-9d30-784d452cbcd4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:09.223658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 519/1044] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:09.229415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:09.332457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29107a88-50ae-4a35-8f54-55f81aa253b7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:09.353290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:09.376881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:09.377496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:09.377807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:09.378084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:09.378812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] RBAC: Adding query filter params (role.id=3b0fe195021a4ab6aa2049dfb765c910, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:09.379123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:09.380544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d69bb7-85c2-48ad-b8b2-5d42c7b25214 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:09.386235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 526/1045] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:09 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=3b0fe195021a4ab6aa2049dfb765c910 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:10.605720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdd62-b841-4936-b1bd-d308cdfa8345 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:10.606387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdd62-b841-4936-b1bd-d308cdfa8345 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:10.606482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdd62-b841-4936-b1bd-d308cdfa8345 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:10.607874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 520/1046] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:10.614215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45c7a-fe31-4b9e-b68e-af8447595e21 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:10.614632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45c7a-fe31-4b9e-b68e-af8447595e21 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:10.614804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b45c7a-fe31-4b9e-b68e-af8447595e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:10.633066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a2b45c7a-fe31-4b9e-b68e-af8447595e21 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:10.634215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a2b45c7a-fe31-4b9e-b68e-af8447595e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:10.701658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 527/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:10.878446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dd277-ec95-4e67-a0d5-d16ea20b975f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:10.878804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dd277-ec95-4e67-a0d5-d16ea20b975f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:10.879007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dd277-ec95-4e67-a0d5-d16ea20b975f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:10.880476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 521/1048] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:10.885576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:10.885848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:10.886123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:10.904154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:10.905343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:10.971603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dd277-ec95-4e67-a0d5-d16ea20b975f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:10.973369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 528/1049] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:10.983120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.079925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dd277-ec95-4e67-a0d5-d16ea20b975f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.113691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.114343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.114621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.114894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.124887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:11.129689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:11.130669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:11.131525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:11.134024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 522/1050] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:11.136909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.184499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.239003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.243251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d817b5-037d-4e1f-a894-6b694c0668c3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.280346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.280962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.281246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.281524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.282222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:11.283092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.284475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.292990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 529/1051] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:11.299042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.318187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.355456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8d67-b329-46f1-be52-df18025caab1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.399461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.400159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.400339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.400578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.401161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.402568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:11.411693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:11.412776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 15:25:11.413610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:11.415180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcebcc-9bdc-468b-8898-dd854de788a3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.418168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 523/1052] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:11.419173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.421838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.453630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.455227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.455508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.455789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] PATH_INFO: `/v3/users/91b0bdc5894446e4a697e475a380423e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.472016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] RBAC: Authorizing `identity:get_user(user_id=91b0bdc5894446e4a697e475a380423e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.473462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.475919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 530/1053] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/users/91b0bdc5894446e4a697e475a380423e => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:11.480962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.527991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a59d80-cb91-4baf-b9db-976d89815ca5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.558045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.564096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.564710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.564993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.565268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.565928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:11.566237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.567601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76617db5-66c4-4eb8-9d04-c299616c7bf7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.572621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 524/1054] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:11.573783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec81a00-3e18-4943-a0ba-37188c86ed98 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.574071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec81a00-3e18-4943-a0ba-37188c86ed98 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.574346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec81a00-3e18-4943-a0ba-37188c86ed98 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.575532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 525/1055] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:11.578049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec81a00-3e18-4943-a0ba-37188c86ed98 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.590182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d95697-dbee-43d8-bcb8-f4cff226be41 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.624332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.624979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.625216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.625486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.654141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.679631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec81a00-3e18-4943-a0ba-37188c86ed98 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.691224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.692667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.713384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.714020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.714300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.714574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.715196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] RBAC: Adding query filter params (type=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:11.715499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.716849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831db3ca-07a9-4a5e-a4c3-3b06355e7448 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.721513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 526/1056] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:11.722653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.722941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.723222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.741096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:11.742308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:11.789344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 531/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 15:25:11.794367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.796612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.808814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 527/1058] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:11.809599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:11.853013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:11.893510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4637c-8ed8-4d05-a5e7-ef210fb2de34 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.924200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816b89-b934-4a5a-836d-4a8e5d668916 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:11.928997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.929595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.929907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.930179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.930838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.932329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.955596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 532/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:25:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:11.976652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:11.977306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:11.977589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:11.977891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:11.979071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:11.980542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:11.982618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 528/1060] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:11.989676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.079713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.105435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcc337-92b6-4528-883f-d8b644ddd5fd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.128534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:12.141850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:12.145803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.146527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.147119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.147244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.147897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=a83c207a6c61490bb7dcf3412c83b471, user.id=91b0bdc5894446e4a697e475a380423e) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:12.148214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:12.149646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:12.155439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 533/1061] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:11 2022] GET /identity/v3/role_assignments?user.id=91b0bdc5894446e4a697e475a380423e&scope.project.id=a83c207a6c61490bb7dcf3412c83b471&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:12.194333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab52f-548f-4faa-ad48-6f8adbfb37b8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.234980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.236365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.236740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.237106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.249089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:12.254324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:12.256502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:12.257584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:12.265575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 529/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:12.269315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.391287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555339-b433-4d64-bf30-816dabb7c56e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.425339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.425968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.426251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.426522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.427178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:12.427480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:12.428843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:12.441938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 534/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:12.443341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.560396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0cd5-b11a-4bac-9add-bfaee7003eaa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.575887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:12.598833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.599476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.599766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.600046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.610421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:12.611883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:12.626631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:12.627576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:12.628465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:12.628465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:12.628465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:12.630032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 530/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:12.633433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.683446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:12.738430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022f022-8a9b-43c6-ad2f-0d15b1260ef1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.772041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.772646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.772927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.773198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.773920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:12.774225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:12.775630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:12.791991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 535/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:12.793524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:12.900958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f5ea-d6b7-49ba-89ad-fd93ee8762fc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:12.936089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:12.936730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:12.937016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:12.937295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:12.946416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:12.947870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:25:12.959828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:12.960753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:12.961732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:12.965734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 531/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:12.966775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.075405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05873452-0744-46ea-a8d2-486e22cf549e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.083409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:13.110773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.111486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.111862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.112226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.113033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:13.113453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.114983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.139032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 536/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:25:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:13.140432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.169620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:13.254059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63c119-c3db-4d25-a64b-be7b401d4a29 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.288549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.289205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.289492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.289807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.301286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.302783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:13.317693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:13.318644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:13.319527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:13.319527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:13.319527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:13.322037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 532/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:13.324748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.430900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856eaba-ce3b-4f67-9bf7-4cdfeb5eec35 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.464044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.464640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.464920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.465196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.465868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:13.466173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.467531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.483424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 537/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:13.484834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.567387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:13.591292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68d8a-7dbb-45da-8a78-35bfee58ef7b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.628452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.629080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.629366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.629641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.636281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.637730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:25:13.649730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:13.650614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:13.651535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:13.654030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 533/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:13.656510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.682989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:13.759573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91a40-c4ef-4e6b-8ec5-b0b668978749 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.792870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.793481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.793794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.794075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.794768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:13.795079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.796512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9bfa0-f24b-436b-8a44-99b396f881ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.813983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 538/1071] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:13.815171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:13.915104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad4e36-ca64-4701-bdc0-e384fe8831a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:13.957286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:13.957901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:13.958181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:13.958451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/3b0fe195021a4ab6aa2049dfb765c910` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:13.960597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=c615724d4e5944169ead59e30e0d04b2, role_id=3b0fe195021a4ab6aa2049dfb765c910)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:13.962059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f7b53-4c87-42e7-9bab-e1cd2ea5c92e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:13.970658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:13.985831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 534/1072] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:25:13 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/c615724d4e5944169ead59e30e0d04b2/roles/3b0fe195021a4ab6aa2049dfb765c910 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:25:14.207649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516b46-3f9f-48f4-a514-10ada90f43d6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.207958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516b46-3f9f-48f4-a514-10ada90f43d6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.208125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516b46-3f9f-48f4-a514-10ada90f43d6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.209750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 539/1073] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:14.215415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb43b-3cd4-48b7-bf04-de1aab37f68f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.215766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb43b-3cd4-48b7-bf04-de1aab37f68f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.215991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfceb43b-3cd4-48b7-bf04-de1aab37f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.236088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bfceb43b-3cd4-48b7-bf04-de1aab37f68f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:14.237338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bfceb43b-3cd4-48b7-bf04-de1aab37f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:14.315695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 535/1074] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:14.330928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e597e-34fb-4f4d-b514-0f2d698a67e6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.331214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e597e-34fb-4f4d-b514-0f2d698a67e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.331486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e597e-34fb-4f4d-b514-0f2d698a67e6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.332941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 540/1075] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:14.337963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.338256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.338535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.357399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:14.358637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:14.428653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 536/1076] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:25:14.586550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e597e-34fb-4f4d-b514-0f2d698a67e6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:14.674024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:14.692260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:14.693838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e597e-34fb-4f4d-b514-0f2d698a67e6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:14.731527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:14.732140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.732423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.732702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.757506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:14.804357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1427-7d49-4e1d-beb3-47315581df13 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:14.807158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:14.808614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:14.840713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:14.841285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:14.841568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:14.841877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:14.854776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:14.859800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:14.861383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:14.862841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 537/1077] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:14.867336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:14.902604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:14.925849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 541/1078] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:14.931101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:14.973763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f9b0b-20ab-4017-91ed-d5257add1bc9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:14.994306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:15.016578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.017454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.017921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.018403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.019221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:15.019685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.021074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.029538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 538/1079] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:15.035119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:15.041933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c646b-e0c4-45b1-b662-4e2451d3f0d8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:15.078542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.079205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.079487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.079760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.089736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.091215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:15.106014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:15.106933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:15.107819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:15.107819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:15.107819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:15.107819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 542/1080] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:15.113816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:15.146071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df70d2-84a8-4c63-af9d-7178345022f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:15.150410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:15.181102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.181768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.182054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.182326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] PATH_INFO: `/v3/users/91b0bdc5894446e4a697e475a380423e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.183471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=91b0bdc5894446e4a697e475a380423e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.184902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.186705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 539/1081] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:15 2022] GET /identity/v3/users/91b0bdc5894446e4a697e475a380423e => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:15.192171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:15.222252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecff7c-58df-4671-af88-c0e731b88d4d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:15.243989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:15.259603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.260244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.260530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.260963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.261614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:15.262447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.263834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.282130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 543/1082] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:15.306808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c6b-fead-44f8-b511-617b046fd9b1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:15.345923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.346494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.346776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.347053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.348221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.349795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b56f-5685-41fc-9931-64a79afd89cc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.352759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 540/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:15 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:15.357980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:15.468941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:15.476376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4cb16-d89c-41f9-88e1-5ec6c0b77398 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:15.511471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:15.512127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:15.512412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:15.512682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471/users/91b0bdc5894446e4a697e475a380423e/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:15.514599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] RBAC: Authorizing `identity:create_grant(project_id=a83c207a6c61490bb7dcf3412c83b471, user_id=91b0bdc5894446e4a697e475a380423e, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:15.516032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e3c5d-16d8-4640-bb81-2b0bc719e84f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:15.529218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 544/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:25:15 2022] PUT /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471/users/91b0bdc5894446e4a697e475a380423e/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:25:16.135933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ea30-5398-48dd-820a-e5c3cdf69246 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:16.136500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ea30-5398-48dd-820a-e5c3cdf69246 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:16.136500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ea30-5398-48dd-820a-e5c3cdf69246 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:16.137870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 541/1085] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:16.143185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:16.143609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:16.143713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:16.161905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:16.163076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:16.229962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 545/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:16.498299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ea30-5398-48dd-820a-e5c3cdf69246 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:16.598077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ea30-5398-48dd-820a-e5c3cdf69246 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:16.614358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:16.632254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:16.632882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:16.633158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:16.633425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:16.643874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:16.648626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:16.649820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:16.650958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 542/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Sep 12 15:25:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:16.658026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:16.739644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:16.755654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844461cc-e31d-4056-b492-c8fceadad188 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:16.788548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:16.789145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:16.789425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:16.789691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:16.790368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:16.790668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:16.792020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:16.799714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 546/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:25:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:16.805438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:16.903429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f1fa9-59e2-42ef-932d-856088cbcb77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:16.936484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:16.937141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:16.937425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:16.937698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:16.948138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:16.949611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:16.963852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:16.964915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:16.965889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:16.965889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:16.965889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:16.965889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 543/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:16.970780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:17.071115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143226c2-cd0d-4f2a-9f4d-b0f5d90dc267 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:17.105162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:17.105798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.106083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.106360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.106997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:17.107299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:17.108660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:17.120087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 547/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:17.125671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:17.153990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:17.227394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650812aa-47f0-48a0-b158-737ac6328a39 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:17.259881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:17.260505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.260783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.261052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.268986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:17.270444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:25:17.281304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:17.287687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:17.288644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 544/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Sep 12 15:25:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:17.289251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:17.299217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:17.388337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93423246-6afd-4436-ab9c-71f687dc967b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:17.421545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:17.422172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.422452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.422725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.423974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:17.424278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:17.425680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92015074-8ace-444e-8b83-61ab5615df1e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:17.442800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 548/1092] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Sep 12 15:25:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:17.448775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:17.536248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:17.549230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c47e7-7589-4df8-8cc3-44f8dbda76cf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:17.587544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:17.588163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.588446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.588722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.655484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:17.656863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:17.675364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414f6a-3b8c-4c65-adbe-fed9b09260a1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.675744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414f6a-3b8c-4c65-adbe-fed9b09260a1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.676107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414f6a-3b8c-4c65-adbe-fed9b09260a1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.677581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 549/1093] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:25:17.682279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.682564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.682839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.701178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:17.702410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:17.744589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:17.758600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 545/1094] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:17.763057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:17.764278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:17.771545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 550/1095] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:25:17.860080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954377-4dbd-4386-93db-1411514ad91f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:17.893794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:17.894420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:17.894707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:17.894986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:17.905127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:17.906668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:17.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:17.933100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:25:17.934509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:17.934509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:25:17.934509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:17.941812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 546/1096] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:17.943085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.039174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.083355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c5d5f-0e52-4068-81ed-06731f5c361b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.095129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.126607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.127520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.127811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.128087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.128737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:18.129041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.130436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.155371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 551/1097] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:18.155371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.176697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feea3f-8172-422d-a3f5-33cb44041988 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.221428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.224731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.225368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.225721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.225977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.243352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:18.249410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:18.250511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:18.251394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:18.257759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 547/1098] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:18.259105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.285598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c121c-7add-4239-9ea7-e5576961b7dd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.327626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.328243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.328526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.328802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.335670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.337115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:25:18.349026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:18.350098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:18.351005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 552/1099] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:25:18.356072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.367042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.372407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2332b74-9740-4d73-ad2f-02cb227f8909 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.408514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.409119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.409404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.409681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.410371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:18.410674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.412042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.420461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 548/1100] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:18.426906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.457918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487696e3-abcd-4d75-8505-1a692adf7cf4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.474400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.492435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.493038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.493335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.493606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.494322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:18.494636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.496108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.507671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 553/1101] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:18.513570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.525958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf93d1-c144-4d72-b21d-2d8044ddddd7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.559874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.560506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.560826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.561105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] PATH_INFO: `/v3/users/91b0bdc5894446e4a697e475a380423e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.562262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] RBAC: Authorizing `identity:get_user(user_id=91b0bdc5894446e4a697e475a380423e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.563720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.565303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 549/1102] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/users/91b0bdc5894446e4a697e475a380423e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:18.571204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.601804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.613434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb47de-d54f-49c7-8f3a-c12e94754d3a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.647040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.647636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.647927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.648190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.648900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] RBAC: Adding query filter params (role.id=3b0fe195021a4ab6aa2049dfb765c910, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=c615724d4e5944169ead59e30e0d04b2) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:18.649204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.650654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.656186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 554/1103] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/role_assignments?user.id=c615724d4e5944169ead59e30e0d04b2&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=3b0fe195021a4ab6aa2049dfb765c910 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:18.674394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79deec-d40e-429c-8c02-0a281fb0e987 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:18.712717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:18.713264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:18.713549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:18.713865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:18.787171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:18.788664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:18.821993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:18.893223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 550/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 15:25:18.899375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:18.941910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:19.005231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86b22-9d0c-4ac6-84b6-c010f14be326 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:19.042006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:19.042642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:19.042927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:19.043204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:19.044315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:19.045781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7507f2-1b47-44b5-bd99-38343716b103 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:19.047432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 555/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:18 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:19.054290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:19.156303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367ab56-029b-40b6-9b4c-5055534608d9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:19.186656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:19.192735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:19.193761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:19.194045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:19.194327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:19.195310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=a83c207a6c61490bb7dcf3412c83b471, user.id=91b0bdc5894446e4a697e475a380423e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:19.195623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:19.197061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a7ce2-33c6-42ac-9c44-6a08ea91dbe2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:19.203040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 551/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:19 2022] GET /identity/v3/role_assignments?user.id=91b0bdc5894446e4a697e475a380423e&scope.project.id=a83c207a6c61490bb7dcf3412c83b471&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:21.355984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be5b9-6682-4c77-b6bc-dff7eb0adce0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:21.357887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be5b9-6682-4c77-b6bc-dff7eb0adce0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:21.357887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84be5b9-6682-4c77-b6bc-dff7eb0adce0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:21.358281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 556/1107] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:21.364228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:21.364671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:21.364977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:21.388167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:21.389356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:21.466622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 552/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:21.731810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84be5b9-6682-4c77-b6bc-dff7eb0adce0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:21.833438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84be5b9-6682-4c77-b6bc-dff7eb0adce0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:21.842074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:21.867515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:21.868140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:21.868420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:21.868692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:21.869823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:21.871249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb99ba-ffc8-470f-90a3-716d3b673e4b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:21.872805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 557/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:21.878668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:21.975180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7827b6-93a1-4523-97fb-d4c564221ee0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:22.000372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:22.012997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:22.013607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:22.013918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:22.014195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:22.014844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:22.016273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3236db4-2aa5-40dc-93b8-67524c0b84d2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:22.037225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 553/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:25:21 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:24.213032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4ea0-f88c-430a-8ce4-f303bd071f11 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:24.213590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4ea0-f88c-430a-8ce4-f303bd071f11 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:24.213590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe4ea0-f88c-430a-8ce4-f303bd071f11 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:24.214969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 558/1111] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:24.220653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:24.221041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:24.221200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:24.239626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:24.240820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:24.306728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 554/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:24.578600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe4ea0-f88c-430a-8ce4-f303bd071f11 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:24.677926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe4ea0-f88c-430a-8ce4-f303bd071f11 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:24.711114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:24.711729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:24.712012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:24.712288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] PATH_INFO: `/v3/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:24.713672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] RBAC: Authorizing `identity:get_role(role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:24.715063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:24.716364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 559/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:24 2022] GET /identity/v3/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:24.722665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:24.819091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99a890-9178-4d5d-b27d-79990356c222 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:24.830224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:24.854742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:24.855373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:24.855659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:24.855938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] PATH_INFO: `/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:24.868787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] RBAC: Authorizing `identity:get_user(user_id=4f89eef779b344a99a0c3cf8f1dc8d69)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:24.870277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:24.871871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 555/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:24 2022] GET /identity/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:24.878428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:24.922721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:24.975562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613fcf-21c7-4940-b67e-8edc6b98d897 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:25.009948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:25.010551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:25.010835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:25.011109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:25.076156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:25.077606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:25.119967 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:25.173394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 560/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:25.179418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:25.286952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade468d2-5264-41ef-b9ca-59e5967c8f15 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:25.322448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:25.323077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:25.323359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:25.323633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:25.324730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:25.326198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7e49f-6272-46a6-bcb1-b4e7da61d528 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:25.327822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 556/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:25 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:25.334159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:25.429863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e154-0a95-4e33-af18-38575c1c9893 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:25.463166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:25.463764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:25.464046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:25.464322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:25.465037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a83c207a6c61490bb7dcf3412c83b471, user.id=4f89eef779b344a99a0c3cf8f1dc8d69) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:25.465353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:25.466807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767af8c-0c59-44b1-895a-243be980a516 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:25.471973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 561/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:25 2022] GET /identity/v3/role_assignments?user.id=4f89eef779b344a99a0c3cf8f1dc8d69&scope.project.id=a83c207a6c61490bb7dcf3412c83b471&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:25:27.558305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27d1f-232c-48f3-90cd-b95da5415a06 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:27.558856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27d1f-232c-48f3-90cd-b95da5415a06 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:27.558856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27d1f-232c-48f3-90cd-b95da5415a06 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:27.560208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 557/1118] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:27.565409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:27.565859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:27.566000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:27.584607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:27.585840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:27.587768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:27.652612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 562/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:27.928683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27d1f-232c-48f3-90cd-b95da5415a06 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:27.967678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:28.038677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27d1f-232c-48f3-90cd-b95da5415a06 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:28.071683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:28.072312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:28.072596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:28.072871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] PATH_INFO: `/v3/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:28.074282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] RBAC: Authorizing `identity:get_role(role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:28.075645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:28.076960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 558/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:27 2022] GET /identity/v3/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:28.083369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:28.177807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef895a-b998-4857-aa4f-bbb9d357c14b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:28.210526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:28.211087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:28.211372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:28.211649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] PATH_INFO: `/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:28.212747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] RBAC: Authorizing `identity:get_user(user_id=4f89eef779b344a99a0c3cf8f1dc8d69)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:28.214217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:28.215773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 563/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:28 2022] GET /identity/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:28.221608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:28.318726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f97c2-e617-48be-b5b1-581dc6a6dbf1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:28.321268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:28.351907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:28.352534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:28.352819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:28.353096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:28.354457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:28.356492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee2d51-6807-4448-8799-80b15084285d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:28.358536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 559/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:28 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:28.369555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:28.379169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:28.466590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1b36-7136-4084-8c49-9ff89cb2badc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:28.498653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:28.499296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:28.499572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:28.499840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471/users/4f89eef779b344a99a0c3cf8f1dc8d69/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:28.502540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] RBAC: Authorizing `identity:create_grant(project_id=a83c207a6c61490bb7dcf3412c83b471, user_id=4f89eef779b344a99a0c3cf8f1dc8d69, role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:28.503964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dfcf5-5d30-45bd-995f-a1b3526bcd9f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:28.515732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 564/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:25:28 2022] PUT /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471/users/4f89eef779b344a99a0c3cf8f1dc8d69/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:25:30.627195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba755bf-6c87-44fc-85f6-c1a3b359bfc2 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:30.627727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba755bf-6c87-44fc-85f6-c1a3b359bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:30.627727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba755bf-6c87-44fc-85f6-c1a3b359bfc2 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:30.629007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 560/1124] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:30.634108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:30.634552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:30.634654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:30.652956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:30.654253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:30.711861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:30.720727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 565/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:30.983189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba755bf-6c87-44fc-85f6-c1a3b359bfc2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:31.083117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba755bf-6c87-44fc-85f6-c1a3b359bfc2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:31.120557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:31.121183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:31.121464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:31.121760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] PATH_INFO: `/v3/roles/8fdb40ef28b84195bd8287a3a8f24651` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:31.123136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] RBAC: Authorizing `identity:get_role(role_id=8fdb40ef28b84195bd8287a3a8f24651)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:31.124502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:31.126418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 561/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:30 2022] GET /identity/v3/roles/8fdb40ef28b84195bd8287a3a8f24651 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:31.132180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:31.228192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff8836-01c6-40e6-b4c2-43ee13d99eeb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:31.261143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:31.261723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:31.262093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:31.262368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] PATH_INFO: `/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:31.263471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] RBAC: Authorizing `identity:get_user(user_id=4f89eef779b344a99a0c3cf8f1dc8d69)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:31.264907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:31.266613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 566/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:31 2022] GET /identity/v3/users/4f89eef779b344a99a0c3cf8f1dc8d69 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:31.272394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:31.314787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:31.372001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01750-44cc-481b-bf19-1978c8dfab5b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:31.405795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:31.406397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:31.406683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:31.406985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:31.473049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:31.474532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:31.513485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:31.570903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 562/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:31.576811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:31.636022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:31.672402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85481bb-9fd6-45dc-bd70-d8071ea55f99 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:31.709103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:31.710208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:31.710710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:31.711021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] PATH_INFO: `/v3/projects/a83c207a6c61490bb7dcf3412c83b471` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:31.712182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=a83c207a6c61490bb7dcf3412c83b471)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:31.714016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1201e-a0eb-4618-8bf7-6efffd49a1c1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:31.716779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 567/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:31 2022] GET /identity/v3/projects/a83c207a6c61490bb7dcf3412c83b471 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:31.724083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:31.821279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea670409-e7b2-4168-84b2-ad13a19287be None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:31.854267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:31.854861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:31.855143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:31.855416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:31.856140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] RBAC: Adding query filter params (role.id=8fdb40ef28b84195bd8287a3a8f24651, scope.project.id=a83c207a6c61490bb7dcf3412c83b471, user.id=4f89eef779b344a99a0c3cf8f1dc8d69) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:31.856447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:31.857896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e6626-897f-490e-b1e9-f13cef1c07b9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:31.864025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 563/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:31 2022] GET /identity/v3/role_assignments?user.id=4f89eef779b344a99a0c3cf8f1dc8d69&scope.project.id=a83c207a6c61490bb7dcf3412c83b471&role.id=8fdb40ef28b84195bd8287a3a8f24651 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:34.009285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9055-9665-46cb-a1c3-0e52a156ff77 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:34.009865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9055-9665-46cb-a1c3-0e52a156ff77 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:34.009865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9055-9665-46cb-a1c3-0e52a156ff77 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:34.011208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 568/1131] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:34.016352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:34.016779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:34.016901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:34.024150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:34.035613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:34.036790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:34.103976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 564/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:34.372341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9055-9665-46cb-a1c3-0e52a156ff77 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:34.472058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9055-9665-46cb-a1c3-0e52a156ff77 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:34.501107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:34.506382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:34.507007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:34.507291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:34.507567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:34.508671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:34.510139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462eb74-bed7-4378-8027-72f62811e2c8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:34.511693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 569/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:34.518103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:34.615113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262b60a-9eb0-462a-8bc7-ee2be358e480 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:34.647894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:34.648434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:34.648718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:34.648997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:34.649639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:34.651096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab65fe-17df-4f79-81e4-80fbfce40608 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:34.655565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:34.678643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 565/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:25:34 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:25:36.827042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4188c4-2ac3-4740-af58-c4acf3029ac0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:36.827620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4188c4-2ac3-4740-af58-c4acf3029ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:36.827620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4188c4-2ac3-4740-af58-c4acf3029ac0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:36.828964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 570/1135] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:36.834347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:36.834765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:36.834905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:36.853150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:36.854363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:36.919270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 566/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:37.185041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4188c4-2ac3-4740-af58-c4acf3029ac0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:37.283873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4188c4-2ac3-4740-af58-c4acf3029ac0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:37.315589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:37.316207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:37.316488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:37.316762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:37.317882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:37.319312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd438-8c05-44e8-abf9-97009b59987b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:37.321120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 571/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:37.326771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:37.428483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e0e61-0d0f-487f-a1ef-692480afeea9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:37.444367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:37.463747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac811b0-4456-49e4-8813-695657031856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:37.464349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac811b0-4456-49e4-8813-695657031856 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:37.464618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac811b0-4456-49e4-8813-695657031856 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:37.464902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac811b0-4456-49e4-8813-695657031856 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:37.465592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac811b0-4456-49e4-8813-695657031856 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:37.467006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac811b0-4456-49e4-8813-695657031856 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:37.488281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 567/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:25:37 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:39.641903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c25c5-79f2-4579-b7f8-895f80b8b1ab None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:39.642432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c25c5-79f2-4579-b7f8-895f80b8b1ab None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:39.642432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c25c5-79f2-4579-b7f8-895f80b8b1ab None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:39.643711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 572/1139] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:39.649071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:39.649494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:39.649664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:39.667506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:39.668706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:39.733868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 568/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:39.999621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c25c5-79f2-4579-b7f8-895f80b8b1ab None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.063994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:40.096592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c25c5-79f2-4579-b7f8-895f80b8b1ab None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:40.129097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:40.129721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:40.130016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:40.130291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:40.140060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:40.145400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:40.146531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:40.147765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 573/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:40.152519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.249235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13689-136b-4d08-94f1-8b3b289549ef None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:40.251310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:40.283966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:40.284565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:40.284849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:40.285135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:40.285813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:40.286122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:40.287493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:40.295270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 569/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:40.301649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.399135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e9351-2343-49cf-944c-50f3a37a5e0c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:40.424386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:40.432110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:40.432733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:40.433015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:40.433286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] PATH_INFO: `/v3/users/8401f5bb56a3442b844d80dd67b46617` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:40.446601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] RBAC: Authorizing `identity:get_user(user_id=8401f5bb56a3442b844d80dd67b46617)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:40.448035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:40.449610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 574/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:40 2022] GET /identity/v3/users/8401f5bb56a3442b844d80dd67b46617 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:40.455420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.553440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cdc18-e8fe-43ae-943d-634b3b59e135 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:40.586174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:40.586777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:40.587057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:40.587330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:40.617248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:40.652795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:40.654306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:40.750318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 570/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:40.756251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.852580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bfa46-852b-45ff-a40c-f1eea48ac508 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:40.885468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:40.886114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:40.886400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:40.886677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] PATH_INFO: `/v3/projects/1f4d0d6159924d4bb4512e2630535f3f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:40.887781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] RBAC: Authorizing `identity:get_project(project_id=1f4d0d6159924d4bb4512e2630535f3f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:40.889214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03070938-18b6-4d75-9501-20cba45d5942 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:40.890997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 575/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:40 2022] GET /identity/v3/projects/1f4d0d6159924d4bb4512e2630535f3f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:40.897071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:40.958814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:40.996159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a31f84-41b1-4bda-a4e8-8885e570395f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:41.029375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:41.030006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:41.030293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:41.030573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:41.031290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=1f4d0d6159924d4bb4512e2630535f3f, user.id=8401f5bb56a3442b844d80dd67b46617) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:41.031603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:41.033605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a071-492d-4119-8d9c-b80977bb535f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:41.039023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 571/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:40 2022] GET /identity/v3/role_assignments?user.id=8401f5bb56a3442b844d80dd67b46617&scope.project.id=1f4d0d6159924d4bb4512e2630535f3f&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:43.149670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cfb38-ee7d-4269-ae3f-d44d59c699c1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:43.150248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cfb38-ee7d-4269-ae3f-d44d59c699c1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:43.150248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cfb38-ee7d-4269-ae3f-d44d59c699c1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:43.151602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 576/1147] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:43.157192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:43.157586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:43.157796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:43.175637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:43.176842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:43.242636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 572/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:43.512465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cfb38-ee7d-4269-ae3f-d44d59c699c1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:43.566937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:43.611119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cfb38-ee7d-4269-ae3f-d44d59c699c1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:43.644326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:43.644942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:43.645218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:43.645485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:43.654998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:43.659761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:43.660904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:43.661961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 577/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:43.666919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:43.723352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:43.763821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f95d88-1bbb-4bfd-9d82-0a1d1ec936e1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:43.796820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:43.797981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:43.798264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:43.798536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:43.799185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:43.799486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:43.800845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:43.808593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 573/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:43.814387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:43.913630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95694a26-e9aa-4571-99b2-398a8b22b538 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:43.927498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:43.947159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:43.947782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:43.948062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:43.948335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] PATH_INFO: `/v3/users/8401f5bb56a3442b844d80dd67b46617` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:43.949433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] RBAC: Authorizing `identity:get_user(user_id=8401f5bb56a3442b844d80dd67b46617)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:43.950903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:43.952468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 578/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:43 2022] GET /identity/v3/users/8401f5bb56a3442b844d80dd67b46617 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:43.958813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:44.055943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e5dc2-5eee-4d78-880b-65ca358f9b50 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:44.075242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:44.089647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:44.090300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:44.090584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:44.090859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] PATH_INFO: `/v3/projects/1f4d0d6159924d4bb4512e2630535f3f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:44.091968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] RBAC: Authorizing `identity:get_project(project_id=1f4d0d6159924d4bb4512e2630535f3f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:44.093396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fae213-6099-4906-8c07-cee27ab72d58 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:44.095598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 574/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:43 2022] GET /identity/v3/projects/1f4d0d6159924d4bb4512e2630535f3f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:44.101524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:44.196014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb88a-0b83-43f5-9460-8c480fae8f47 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:44.229442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:44.230119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:44.230398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:44.230672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] PATH_INFO: `/v3/projects/1f4d0d6159924d4bb4512e2630535f3f/users/8401f5bb56a3442b844d80dd67b46617/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:44.232545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] RBAC: Authorizing `identity:create_grant(project_id=1f4d0d6159924d4bb4512e2630535f3f, user_id=8401f5bb56a3442b844d80dd67b46617, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:44.234007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca324f-2dcc-4c08-9362-8f174f17423d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:44.248704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 579/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:25:44 2022] PUT /identity/v3/projects/1f4d0d6159924d4bb4512e2630535f3f/users/8401f5bb56a3442b844d80dd67b46617/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:25:46.335691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0c25-3027-4b33-952c-4d06fa5bca3a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:46.336328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0c25-3027-4b33-952c-4d06fa5bca3a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:46.336328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0c25-3027-4b33-952c-4d06fa5bca3a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:46.337589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 575/1154] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:46.342894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:46.343320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:46.343442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:46.361818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:46.363015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:46.397390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:46.429134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 580/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:46.700219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b0c25-3027-4b33-952c-4d06fa5bca3a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:46.800578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b0c25-3027-4b33-952c-4d06fa5bca3a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:46.833081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:46.833727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:46.834014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:46.834290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:46.844084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:25:46.849399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:46.850604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:25:46.851705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 576/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:46.856529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:46.951392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2beaa-9a5f-4503-9ab6-b374caa7c17b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:46.984352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:46.984969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:46.985251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:46.985523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:46.986214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:46.986519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:46.987880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:46.996042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 581/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:47.001889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:47.050953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:47.100673 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334ffa8-a1fc-4e60-a857-d841805ec849 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:47.133063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:47.133678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:47.133987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:47.134258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] PATH_INFO: `/v3/users/8401f5bb56a3442b844d80dd67b46617` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:47.135356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] RBAC: Authorizing `identity:get_user(user_id=8401f5bb56a3442b844d80dd67b46617)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:47.136778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:47.138357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 577/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:25:47 2022] GET /identity/v3/users/8401f5bb56a3442b844d80dd67b46617 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:47.144565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:47.178066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:47.243840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde9a0-0327-4b8d-a0a8-ca626b4b574f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:47.280542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:47.280542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:47.280542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:47.280542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:47.346998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:47.348441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:47.379889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:47.443920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 582/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:47.451639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:47.547972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3595630-356c-4a34-8823-1fb1396ee131 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:47.564342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:47.582544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:47.583162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:47.583444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:47.583718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] PATH_INFO: `/v3/projects/1f4d0d6159924d4bb4512e2630535f3f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:47.584814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] RBAC: Authorizing `identity:get_project(project_id=1f4d0d6159924d4bb4512e2630535f3f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:47.586280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7bff4-ed78-4945-9aa1-d7257a0277c5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:47.587881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 578/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:25:47 2022] GET /identity/v3/projects/1f4d0d6159924d4bb4512e2630535f3f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:47.593917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:47.691526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c932674-841c-4a46-b05c-58d2b03a75dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:47.724739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:47.725767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:47.726283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:47.726894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:47.728012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=1f4d0d6159924d4bb4512e2630535f3f, user.id=8401f5bb56a3442b844d80dd67b46617) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:47.728333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:47.729795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e04cd-7de5-4012-aa22-5f0017b67fef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:47.731560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:47.737107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 583/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:25:47 2022] GET /identity/v3/role_assignments?user.id=8401f5bb56a3442b844d80dd67b46617&scope.project.id=1f4d0d6159924d4bb4512e2630535f3f&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:49.904153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cef6d-26a6-401e-8711-0c28b5a3dd68 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:49.904689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cef6d-26a6-401e-8711-0c28b5a3dd68 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:49.904689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cef6d-26a6-401e-8711-0c28b5a3dd68 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:49.906087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 579/1162] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:49.911185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:49.911570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:49.911748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:49.929991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:49.931183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:49.995749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 584/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:50.260019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cef6d-26a6-401e-8711-0c28b5a3dd68 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:50.359278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cef6d-26a6-401e-8711-0c28b5a3dd68 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:50.398326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:50.398927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:50.399205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:50.399474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:50.449121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:50.465136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:50.466607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:50.565552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 580/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:25:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:25:50.571730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:50.666057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea3e6-92e2-4689-8d2f-335edf027189 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:50.691376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:50.699620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:50.700264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:50.700532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:50.700804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:50.708203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:50.709655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-02d404ae-c0ee-466e-bbca-af0eecef9e8f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:25:50.721631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:25:50.722863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:25:50.723931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 585/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:25:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:25:50.728709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:50.808872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:50.824807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa4294-7015-4893-82bc-730d5686edf1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:50.858756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:50.859355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:50.859635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:50.859939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:50.860572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:25:50.860877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:50.862270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece97cd9-c568-48ca-873e-ed121f0f412f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:50.869321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 581/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Sep 12 15:25:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:52.992646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8494-4835-490d-b58f-0181a30da40d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:52.993197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8494-4835-490d-b58f-0181a30da40d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:52.993197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8494-4835-490d-b58f-0181a30da40d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:52.994738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 586/1167] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:52.999863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e336f4-c420-48e5-951f-20909bc1212b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:53.000340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e336f4-c420-48e5-951f-20909bc1212b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:53.001006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e336f4-c420-48e5-951f-20909bc1212b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:53.018776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-60e336f4-c420-48e5-951f-20909bc1212b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:53.019950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-60e336f4-c420-48e5-951f-20909bc1212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:53.084671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 582/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:53.350942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd8494-4835-490d-b58f-0181a30da40d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:53.461226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd8494-4835-490d-b58f-0181a30da40d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:53.494605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:53.495191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:53.495474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:53.495746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:53.496404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:53.497778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd0c5-0b47-4a8e-b0b0-c417d77ff100 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:53.515136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 587/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Sep 12 15:25:53 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:55.682665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b485-5d75-491e-a05c-7dceb922b844 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:55.683196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b485-5d75-491e-a05c-7dceb922b844 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:55.683196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8b485-5d75-491e-a05c-7dceb922b844 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:55.684466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 583/1170] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:55.690097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:55.690515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:55.690644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:55.709021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:55.710270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:55.753898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:55.777081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 588/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:56.051289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8b485-5d75-491e-a05c-7dceb922b844 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:56.132866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:56.154850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8b485-5d75-491e-a05c-7dceb922b844 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:56.187479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:56.188658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:56.188937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:56.189206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] PATH_INFO: `/v3/domains/0da89eaf1ac34fb492f619fddcbf760d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:56.199505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0da89eaf1ac34fb492f619fddcbf760d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:56.200945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387361a-93ea-4b09-8221-cce970b34b81 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:56.202525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 584/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Sep 12 15:25:56 2022] GET /identity/v3/domains/0da89eaf1ac34fb492f619fddcbf760d => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:56.208312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:56.304809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fd50c-0216-4733-8fac-d425630bb2eb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:56.336942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:56.337535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:56.337840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:56.338112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:56.338742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:56.340159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2c0a9-5f29-453b-b277-79e2927151c9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:56.366534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 589/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:25:56 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:25:58.503009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734470e9-8262-49b0-9e43-13b02593b51b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:58.503555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734470e9-8262-49b0-9e43-13b02593b51b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:58.503555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734470e9-8262-49b0-9e43-13b02593b51b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:58.504882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 585/1174] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:25:58.510299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:58.510744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:58.510886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:58.529280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:25:58.530498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:25:58.553463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:25:58.596367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 590/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:25:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:25:58.863091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734470e9-8262-49b0-9e43-13b02593b51b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:58.963619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734470e9-8262-49b0-9e43-13b02593b51b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:58.996858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:58.997485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:58.997799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:58.998081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] PATH_INFO: `/v3/domains/0da89eaf1ac34fb492f619fddcbf760d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:58.999199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0da89eaf1ac34fb492f619fddcbf760d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:59.001222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24ae93-228c-4d96-a3d6-db588c2a7708 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:59.002812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 586/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Sep 12 15:25:58 2022] GET /identity/v3/domains/0da89eaf1ac34fb492f619fddcbf760d => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:25:59.008577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:25:59.102492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbdf1c-2e6a-435a-8bfb-b449bc1f7a45 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:25:59.135542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:25:59.136087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:25:59.136373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:25:59.136652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:25:59.137302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:25:59.138764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f56074-f0a8-459c-915c-7b8a5157aa90 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:25:59.162776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 591/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:25:59 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:26:01.331420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d019e2-efd2-48b3-96eb-38b6ae1e96fb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:01.331961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d019e2-efd2-48b3-96eb-38b6ae1e96fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:01.331961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d019e2-efd2-48b3-96eb-38b6ae1e96fb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:01.333301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 587/1178] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:26:01.338388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:01.338800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:01.338936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:01.356561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:26:01.357765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:26:01.368587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:01.423275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 592/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:26:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:26:01.689434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d019e2-efd2-48b3-96eb-38b6ae1e96fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:01.698657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:01.788534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d019e2-efd2-48b3-96eb-38b6ae1e96fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:01.821381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:01.822037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:01.822322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:01.822598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:01.832893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:26:01.837695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:26:01.838780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:26:01.839618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 588/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:26:01.844702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:01.940245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d27026-d8d8-45ef-b906-025785ff072f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:01.972858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:01.973456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:01.973762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:01.974041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:01.974726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:26:01.975030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:01.976396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:01.984064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 593/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:26:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:01.990918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:02.056522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:02.091021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5d50-fb38-463e-847f-632cedf2ee7f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:02.123783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:02.124394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:02.124674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:02.124945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] PATH_INFO: `/v3/users/87c699714ca84e49aa99f569c4e99498` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:02.137986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] RBAC: Authorizing `identity:get_user(user_id=87c699714ca84e49aa99f569c4e99498)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:02.139420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:02.141493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 589/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:26:01 2022] GET /identity/v3/users/87c699714ca84e49aa99f569c4e99498 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:02.147283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:02.158695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:02.243914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30405-b3df-48b3-ae8c-7d5bfc7591ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:02.275965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:02.276551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:02.276829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:02.277099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:02.344791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:02.346358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:02.355486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:02.441295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 594/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:26:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:26:02.447473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:02.544351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a9ad-3a33-428e-b810-e01dbd4e6160 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:02.567436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:02.579081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:02.579704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:02.579988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:02.580263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] PATH_INFO: `/v3/projects/98e8cb4c030b41618d1a7e01f40297a7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:02.581366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] RBAC: Authorizing `identity:get_project(project_id=98e8cb4c030b41618d1a7e01f40297a7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:02.582838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ccc24-fcd4-4a20-8920-aabee64bb94a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:02.584440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 590/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:26:02 2022] GET /identity/v3/projects/98e8cb4c030b41618d1a7e01f40297a7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:02.590437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:02.684940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dff2e2-b280-4662-8fcd-c8517d70ebc6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:02.694361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:02.718528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:02.719152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:02.719436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:02.719713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:02.720430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=98e8cb4c030b41618d1a7e01f40297a7, user.id=87c699714ca84e49aa99f569c4e99498) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:26:02.720741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:02.722200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6936f-5f4b-4560-9473-6af1834b84c3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:02.727329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 595/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:26:02 2022] GET /identity/v3/role_assignments?user.id=87c699714ca84e49aa99f569c4e99498&scope.project.id=98e8cb4c030b41618d1a7e01f40297a7&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:04.843596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223dd3-eefd-4a55-a364-7560acabcab1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:04.844681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223dd3-eefd-4a55-a364-7560acabcab1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:04.844681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223dd3-eefd-4a55-a364-7560acabcab1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:04.846032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 591/1186] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:26:04.851257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:04.851655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:04.851814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:04.870164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:26:04.871354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:26:04.935201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 596/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:26:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:26:05.207040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01223dd3-eefd-4a55-a364-7560acabcab1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:05.306285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01223dd3-eefd-4a55-a364-7560acabcab1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:05.340610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:05.341218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:05.341498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:05.341873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:05.351707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:26:05.356562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:26:05.357763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:26:05.358890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 592/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:26:05.363874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:05.460584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261d33-63af-44ed-86a3-10ae86aff217 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:05.462011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:05.495158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:05.495758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:05.496045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:05.496322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:05.496984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:26:05.497292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:05.498697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:05.507025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 597/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:05.512871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:05.564961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:05.611648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75278f-f03d-4b2a-9088-f5cfedcb9651 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:05.644680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:05.645293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:05.645572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:05.645874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] PATH_INFO: `/v3/users/87c699714ca84e49aa99f569c4e99498` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:05.646979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] RBAC: Authorizing `identity:get_user(user_id=87c699714ca84e49aa99f569c4e99498)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:05.648394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:05.649964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 593/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:26:05 2022] GET /identity/v3/users/87c699714ca84e49aa99f569c4e99498 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:05.655818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:05.752312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e16dc-c898-48e4-8729-553d68412df2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:05.785064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:05.785681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:05.785994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:05.786269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] PATH_INFO: `/v3/projects/98e8cb4c030b41618d1a7e01f40297a7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:05.787373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] RBAC: Authorizing `identity:get_project(project_id=98e8cb4c030b41618d1a7e01f40297a7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:05.788801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e10dd1-80e2-4846-a1ab-835bc7f42e06 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:05.790440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 598/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:26:05 2022] GET /identity/v3/projects/98e8cb4c030b41618d1a7e01f40297a7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:05.796155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:05.892194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54c12b-ac89-4778-bdad-81d7819a9978 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:05.912316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:05.926518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:05.927168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:05.927454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:05.927731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] PATH_INFO: `/v3/projects/98e8cb4c030b41618d1a7e01f40297a7/users/87c699714ca84e49aa99f569c4e99498/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:05.929614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] RBAC: Authorizing `identity:create_grant(project_id=98e8cb4c030b41618d1a7e01f40297a7, user_id=87c699714ca84e49aa99f569c4e99498, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:05.931065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d94c38-17b5-41d7-b36d-4c319c8f5050 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:05.943894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 594/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:26:05 2022] PUT /identity/v3/projects/98e8cb4c030b41618d1a7e01f40297a7/users/87c699714ca84e49aa99f569c4e99498/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:26:08.028357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3c13-cb28-40ca-b70d-4a46bb9639d2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.028892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3c13-cb28-40ca-b70d-4a46bb9639d2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.028892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3c13-cb28-40ca-b70d-4a46bb9639d2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.030274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 599/1193] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:26:08.036102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.036544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.036636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.054414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:26:08.055595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:26:08.120047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 595/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:26:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:26:08.385224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3c13-cb28-40ca-b70d-4a46bb9639d2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:08.416130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:08.484542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3c13-cb28-40ca-b70d-4a46bb9639d2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:08.517041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:08.517654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.517958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.518226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.527879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:26:08.532782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:26:08.533890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:26:08.535105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 600/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:26:08.539711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:08.636151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10d25-1396-49b4-afa3-28a4f0cc5e04 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:08.647080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:08.671189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:08.671784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.672604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.672877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.673537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:26:08.673872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:08.675237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:08.682986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 596/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:08.688805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:08.776011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:08.785362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0fe70-4239-41ef-ba15-219d9f71301f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:08.817146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:08.817775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.818057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.818329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] PATH_INFO: `/v3/users/87c699714ca84e49aa99f569c4e99498` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.819418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] RBAC: Authorizing `identity:get_user(user_id=87c699714ca84e49aa99f569c4e99498)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:08.821102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:08.822658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 601/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Sep 12 15:26:08 2022] GET /identity/v3/users/87c699714ca84e49aa99f569c4e99498 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:08.828724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:08.924857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cf284-f8d4-45c6-aff7-fa202a42c52c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:08.957632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:08.958253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:08.958537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:08.958812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:08.998419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:09.024588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:09.026084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:09.121431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 597/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:26:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:26:09.127322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:09.222362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7548-1ffb-4219-bec8-c0af25113f12 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:09.254847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:09.255460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:09.255743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:09.256019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] PATH_INFO: `/v3/projects/98e8cb4c030b41618d1a7e01f40297a7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:09.257125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] RBAC: Authorizing `identity:get_project(project_id=98e8cb4c030b41618d1a7e01f40297a7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:09.258587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08bab0-7174-4d78-8c95-28131cb936bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:09.260199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 602/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Sep 12 15:26:09 2022] GET /identity/v3/projects/98e8cb4c030b41618d1a7e01f40297a7 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:09.266210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:26:09.335699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:09.369127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fd1e1-e809-45c8-b02b-b04ec3176d27 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:26:09.402304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:26:09.402900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:26:09.403181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:26:09.403456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:26:09.404172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=98e8cb4c030b41618d1a7e01f40297a7, user.id=87c699714ca84e49aa99f569c4e99498) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:26:09.404484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:26:09.406006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9821d-337f-41a7-83b1-c58f3b1bd593 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:26:09.412142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 598/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:26:09 2022] GET /identity/v3/role_assignments?user.id=87c699714ca84e49aa99f569c4e99498&scope.project.id=98e8cb4c030b41618d1a7e01f40297a7&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:26:19.250927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:26:19.409560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=32 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:04.638575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4fff-c074-4483-8f83-2412b7c8e4c1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:04.638575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4fff-c074-4483-8f83-2412b7c8e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:04.638575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4fff-c074-4483-8f83-2412b7c8e4c1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:04.641492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 603/1201] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:04.647379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:04.647854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:04.647962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:04.674585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:04.676069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:04.792751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 599/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:05.072440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b4fff-c074-4483-8f83-2412b7c8e4c1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:05.211586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b4fff-c074-4483-8f83-2412b7c8e4c1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:05.255963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:05.256626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:05.256947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:05.257229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:05.271691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:05.273227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:05.290154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:05.291413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:05.292974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:05.292974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:05.292974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:05.294625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 604/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:05.301364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:05.434119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fbe23-e5b3-4681-bd09-812169a91582 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:05.475946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:05.482607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:05.483221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:05.483522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:05.484503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:05.485347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:05.485730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:05.487616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9aad2-d0f4-4fd1-a0a6-0052ca4a6a84 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:05.503699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 600/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:05.509333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:05.575851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:05.641749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b512ba-18d1-490c-b4c2-0a0c1273ff14 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:05.681478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:05.682527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:05.682527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:05.682928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:05.683662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:05.685246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473e0ed-854c-4949-9b47-ee7631136aae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:05.714421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 605/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Sep 12 15:27:05 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:27:08.026783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b5e4-5b8b-470f-93ef-8cd9f8b06293 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:08.027510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b5e4-5b8b-470f-93ef-8cd9f8b06293 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:08.027510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b5e4-5b8b-470f-93ef-8cd9f8b06293 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:08.028835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 601/1206] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:08.034610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:08.035043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:08.035182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:08.056339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:08.057641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:08.135430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 606/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 15:27:08.405781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417b5e4-5b8b-470f-93ef-8cd9f8b06293 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:08.531553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417b5e4-5b8b-470f-93ef-8cd9f8b06293 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:08.574493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:08.575426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:08.575797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:08.576208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:08.592661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:08.600848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:08.601875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:08.606342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 602/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:27:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:08.608529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:08.664941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:08.747737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d53fb-a79c-4eff-82d8-9a3036811b0f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:08.792118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:08.792782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:08.793152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:08.793557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:08.794302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:08.794641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:08.796759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:08.810579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 607/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:27:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:08.812811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:08.880221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:08.940647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331809-a53d-4cdc-af03-a033ef5b05f7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:08.985041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:08.985689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:08.986069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:08.986325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:08.997864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:09.000508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:09.016908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:09.018405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:09.019675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:09.019675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:09.019675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:09.021897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 603/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:09.023923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:09.150746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:09.156982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0bba6-05a2-431f-b444-f46d70582bec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:09.207220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:09.208151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:09.208500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:09.208860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:09.209682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:09.210080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:09.212068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:09.232746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 608/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:09.236253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:09.352035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52577c6-bf7d-4934-8572-9badf5b148b3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:09.353759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:09.393985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:09.394635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:09.394921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:09.395299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:09.404503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:09.406600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:09.418873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:09.419901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:09.421888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 604/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:09.426322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:09.547114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4394310-2765-4f95-a510-95cc7318e6cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:09.592669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:09.593956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:09.594251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:09.594569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:09.595486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:09.595797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:09.597227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:09.617778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 609/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:09.623563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:09.746561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b753e-31e5-4aca-adf6-fb4fffa471e7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:09.789816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:09.790503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:09.790819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:09.791068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:09.829131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:09.874895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:09.876466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:09.984453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 605/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:27:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:27:09.990533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:10.012064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:10.112497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e3493-591c-4dcf-928f-2f610ed0c712 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:10.151871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:10.152660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:10.153058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:10.153459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:10.165393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:10.166905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:10.182321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:10.183369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:10.184282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:10.184282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:10.184282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:10.190164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 610/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:10.190295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:10.263469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:10.304932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758957-493f-48b5-934c-16d733eae5bf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:10.341503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:10.342132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:10.342415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:10.342692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:10.343914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:10.344189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:10.345555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:10.357486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 606/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:10.363412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:10.443614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:10.477923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaee24-e31e-45c0-bda3-a65d7165cc38 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:10.515281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:10.515939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:10.516225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:10.516504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:10.523708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:10.525161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:10.539212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:10.540337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:10.546079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 611/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:10.546612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:10.658157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f479e-eae6-4525-8c17-b8238021668c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:10.681861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:10.695832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:10.696480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:10.697040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:10.697369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:10.698140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:10.698452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:10.700016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac686-079c-4430-8466-313bb9e1f845 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:10.714413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 607/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:10.720744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:10.830289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a818cd-cfa5-4382-bcc5-90885e19f5d4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:10.862947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:10.867639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:10.868298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:10.868615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:10.869160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:10.870079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=45b541633f92420e9b118c25775d646a) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:10.870405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:10.871843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9425ec6-a409-48a8-a60d-82a7dcbccec5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:10.878131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 612/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:27:10 2022] GET /identity/v3/role_assignments?user.id=45b541633f92420e9b118c25775d646a&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:13.108270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265cfa-16a5-4583-9680-30a64043fdea None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:13.109384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265cfa-16a5-4583-9680-30a64043fdea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:13.109384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5265cfa-16a5-4583-9680-30a64043fdea None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:13.110795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 608/1220] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:13.116329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:13.116848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:13.117102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:13.136668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:13.137911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:13.207547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 613/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:13.486562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5265cfa-16a5-4583-9680-30a64043fdea None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:13.607602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5265cfa-16a5-4583-9680-30a64043fdea None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:13.646350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:13.647116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:13.647503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:13.647765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:13.660928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:13.666070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:13.667474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:13.668612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 609/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:27:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:13.673819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:13.782618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bfe0c-6912-40ab-987d-b789f3a64129 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:13.808633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:13.821388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:13.822367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:13.822685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:13.823002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:13.823736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:13.824080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:13.825647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:13.835915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 614/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:27:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:13.842182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:13.896184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:13.963937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a9a15-0e3a-4847-bd3f-d832894a1b01 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.006410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.007092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.007379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.007655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.020394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.021945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:14.041175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:14.042549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:14.043674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:14.043674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:14.043674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:14.043674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 610/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:14.050667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:14.165166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3200-38af-41e9-98fa-e5c51232e2b5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.206785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.207633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.207997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.208449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.209218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:14.209581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.211378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.231084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 615/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:14.232742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:14.342807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:14.351121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa2bcc-a632-4816-a583-bc7f8cbc2fc7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.390648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.391467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.391832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.392122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.401542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.403404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:14.417938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:14.419317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:14.426814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 611/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:14.427544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:14.486852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:14.545180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3995-5c0f-4f48-b5cf-696ff2f6e2a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.587047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.587676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.588028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.588322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.589115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:14.589421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.590882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.610010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 616/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:14.616459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:14.735729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520badd-ab4a-4447-a615-87f4d73064b2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.772724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.773362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.773645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.773954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.785672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.787207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:14.802991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:14.804125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:14.805038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:14.805038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:14.805038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:14.805038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 612/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:14.809961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:14.928245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d2725-0d5f-498d-9e06-1389295847f9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:14.930692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:14.967705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:14.968346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:14.968631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:14.968910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:14.969552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:14.969892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:14.971270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:14.983387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 617/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:14.989556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:15.000890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:15.104686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e1f75-2928-45aa-a5ba-b7e1e1cc2c14 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:15.140750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:15.141473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:15.141784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:15.142063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:15.151318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:15.152955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:15.166022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:15.167274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:15.168603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 613/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:15.173936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:15.283107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10679a0f-ac44-4972-87e1-9bd54dda41f2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:15.322502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:15.323238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:15.323563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:15.323907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:15.324764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:15.325155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:15.326884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b50b33-38e1-4a94-9f50-298bde3f7553 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:15.342591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 618/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:15.349149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:15.425642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:15.467639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944ba5d-9205-48b7-a456-841d73677566 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:15.503489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610684a2-e963-492c-9680-30fd60198170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:15.504279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610684a2-e963-492c-9680-30fd60198170 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:15.504671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610684a2-e963-492c-9680-30fd60198170 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:15.505087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610684a2-e963-492c-9680-30fd60198170 None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/45b541633f92420e9b118c25775d646a/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:15.521534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610684a2-e963-492c-9680-30fd60198170 None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=45b541633f92420e9b118c25775d646a, role_id=c9ea3e031d1b4bc69616ef4b5c64cfa7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:15.523125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610684a2-e963-492c-9680-30fd60198170 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:15.536578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 614/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:27:15 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/45b541633f92420e9b118c25775d646a/roles/c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:27:17.906992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf039-0182-4c47-becb-6c9d27afc1e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:17.907533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf039-0182-4c47-becb-6c9d27afc1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:17.907533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf039-0182-4c47-becb-6c9d27afc1e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:17.908880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 619/1233] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:17.915556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:17.915969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:17.916135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:17.938244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:17.939568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:18.023058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 615/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:18.329433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbf039-0182-4c47-becb-6c9d27afc1e5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:18.358362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:18.446450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbf039-0182-4c47-becb-6c9d27afc1e5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:18.484225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:18.485131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:18.485417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:18.485756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:18.496786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:18.501855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:18.503064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:18.504150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 620/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Sep 12 15:27:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:18.509170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:18.563270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:18.630651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661208-0c47-4dc6-a95c-a8b3a1108fad None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:18.677580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:18.678381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:18.678677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:18.679065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:18.679821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:18.680139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:18.681671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:18.691615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 616/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Sep 12 15:27:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:18.697987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:18.790334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:18.808051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfa260-1547-4ce8-b173-a3684b95fc83 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:18.848110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:18.848760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:18.849197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:18.849479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:18.861128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:18.862454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:18.878352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:18.879568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:18.880467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:18.880467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:18.880467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:18.885108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 621/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:18.885865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:18.998071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67101991-94df-42b5-96e8-a18f4e542e45 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:19.009028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:19.038477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:19.039290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:19.039683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:19.040183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:19.040851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:19.041298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:19.042915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:19.059568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 617/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:19.062692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:19.167690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1732-0f22-4a65-b548-6df3b22a7528 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:19.209983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:19.210790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:19.211145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:19.211466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:19.214828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:19.220409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:19.222143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:19.234438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:19.235455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:19.236414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 622/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:19.241778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:19.356178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bc33-7899-4644-9ad5-15b9beaef545 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:19.405142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:19.406101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:19.406512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:19.406888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:19.408062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:19.408354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:19.410148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:19.433891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 618/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:19.440532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:19.556244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2e85-f3a2-43f0-b6fd-f7e0a2207281 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:19.601960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:19.602826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:19.603287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:19.603678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:19.677952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:19.685726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:19.687316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:19.801081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 623/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:27:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 15:27:19.807135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:19.835787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:19.913514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c6bbc-fe3a-484c-a3af-882f5ccc3210 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:19.954407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:19.955208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:19.955654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:19.956041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:19.972403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:19.974329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:19.993792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:19.994879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:19.995801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:19.995801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:19.995801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:19.995801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 619/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:20.000912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:20.109800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:20.116646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07872455-25b0-40dd-af26-a7f1b1c105fb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:20.151607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:20.152260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:20.152546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:20.152823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:20.153472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:20.153820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:20.155349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:20.168135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 624/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:20.174002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:20.265264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:20.293589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f670a-f095-4540-9962-ad807ac05bd9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:20.333389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:20.334055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:20.334339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:20.334611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:20.341286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:20.342765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:20.354703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:20.355765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:20.356651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 620/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:20.361961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:20.476810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38edd8e-c804-4823-94ed-ddc90e37ae08 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:20.510669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:20.511274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:20.511557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:20.511827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:20.512519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:20.512829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:20.514292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c2f3e-7d90-4086-9c64-52a121e784e9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:20.526710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 625/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:20.532768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:20.648187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286466-3adc-455b-9754-bb82a1a3ee2b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:20.686016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:20.686662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:20.687434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:20.687705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:20.688425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] RBAC: Adding query filter params (role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=45b541633f92420e9b118c25775d646a) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:20.688732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:20.690832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c94cf-f967-461a-bf40-79e89645ae6f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:20.697717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 621/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:27:20 2022] GET /identity/v3/role_assignments?user.id=45b541633f92420e9b118c25775d646a&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c9ea3e031d1b4bc69616ef4b5c64cfa7 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:23.388797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96458-aca4-4e04-b2a7-09a18a84aa07 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:23.389313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96458-aca4-4e04-b2a7-09a18a84aa07 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:23.389406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96458-aca4-4e04-b2a7-09a18a84aa07 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:23.391798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 626/1247] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:23.397780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:23.398234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:23.398508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:23.406262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:23.419912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:23.421261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:23.492544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 622/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:23.807783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96458-aca4-4e04-b2a7-09a18a84aa07 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:23.827480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:23.916553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96458-aca4-4e04-b2a7-09a18a84aa07 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:23.951097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:23.951725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:23.952080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:23.952893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:23.963646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:23.969320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:23.971044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:23.972233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 627/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:27:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:23.976838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.099168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc8f35-cda4-470f-9b2b-ba0ba16aacaf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:24.134938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:24.141122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:24.141799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:24.142049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:24.142324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:24.142994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:24.143301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:24.144687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:24.153258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 623/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:24.160880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.253638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:24.280785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2b21-239d-4655-8a67-d98e1c51a9f7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:24.319544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:24.320235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:24.320530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:24.320825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:24.332202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:24.333809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:24.351199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:24.352261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:24.353139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:24.353139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:24.353139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:24.353139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 628/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:24.358356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.482809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dccd0-1bb4-47f1-9b3f-f0f056af7e89 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:24.521015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:24.521641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:24.521953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:24.522226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:24.522886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:24.523191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:24.524747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:24.538904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 624/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:24.544789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.651001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31045e84-d05a-4b67-8962-b1f70c726b2d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:24.678464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:24.687488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:24.688138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:24.688417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:24.688689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:24.697330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:24.698850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:24.711140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:24.712311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:24.713257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 629/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:24.718423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.776848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:24.824929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65fd7d-f772-4647-88f9-ef1203c5d513 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:24.862100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:24.862724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:24.863006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:24.863289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:24.864023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:24.864333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:24.865801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:24.884805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 625/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:24.890834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:24.991290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e12e-51e1-4621-a6de-22d992f821cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.029146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.029875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.030791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.031019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.091207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:25.104907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.106400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.205248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 630/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:27:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 15:27:25.211017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:25.309007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905d5c8-c9f9-44a1-9c73-a887ba33d743 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.321776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:25.344325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.344949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.345234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.345510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.355329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.356772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:25.372153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:25.373241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:25.374218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:25.374218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:25.374218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:25.374218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 626/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:25.378971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:25.462656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:25.475082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef4a1a-987f-4353-ab90-472f8ecef7e8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.506556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.507148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.507427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.507694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.508315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:25.508615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.510079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.521191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 631/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:25.526687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:25.624802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0172b-d61b-4154-99ad-a4bd68c94b6a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.657421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.658061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.658345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.658620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.665083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.666551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:25.677572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:25.678638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:25.679761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 627/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:25.684752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:25.780865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313eaf2-44bc-4360-b48c-b10700ebb090 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.816484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.817076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.817352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.817621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.818326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:25.818630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.820046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb390a-218f-45e7-a92b-c6202092ac73 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.831870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 632/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:27:25.838001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:25.865503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:25.937960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cec434-3023-4c48-bacd-68e2626575e0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:25.971117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:25.971734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:25.972014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:25.972283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:25.972995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=45b541633f92420e9b118c25775d646a) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:25.973329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:25.974790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d072-67d8-4c36-bb94-d7e5dd646610 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:25.979983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 628/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:27:25 2022] GET /identity/v3/role_assignments?user.id=45b541633f92420e9b118c25775d646a&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:28.079591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe398a-6853-4835-8ae2-635830abcdf7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:28.080125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe398a-6853-4835-8ae2-635830abcdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:28.080254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe398a-6853-4835-8ae2-635830abcdf7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:28.081545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 633/1261] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:27:28.086741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:28.087142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:28.087297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:28.105029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:28.106258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:28.172959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 629/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:28.439022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe398a-6853-4835-8ae2-635830abcdf7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:28.450051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:28.537151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe398a-6853-4835-8ae2-635830abcdf7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:28.569966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:28.570582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:28.570862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:28.571136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:28.581227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:28.586056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:28.587142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:28.588367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 634/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:27:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:28.593582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:28.612515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:28.690544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44f00b-91e6-4492-bb9a-713690092a4b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:28.723112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:28.723707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:28.723987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:28.724260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:28.724907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:28.725207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:28.726588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:28.734332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 630/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:27:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:28.739923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:28.810038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:28.842934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece075b-8836-4735-a1ba-aaad159091f1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:28.875435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:28.876046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:28.876322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:28.876588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:28.886097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:28.887524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:28.902323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:28.903459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:28.904377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:28.904377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:28.904377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:28.904377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 635/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:28.909222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:28.987647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:29.009216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f47f4-fa01-416d-9faa-d7a85d88a622 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.050456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.051195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.051522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.051814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.052559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:29.052929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.054615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.069404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 631/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:29.077279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.173199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb762ef2-e079-4eda-8004-e07c663abfb9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.192324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:29.206245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.206856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.207139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.207414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.214523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.215955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:29.227057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:29.228075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:29.229098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 636/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:29.234449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.330855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578a21-6fdf-48cc-8649-5eea479a16a2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.364136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.364733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.365013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.365284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.366032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:29.366337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.367760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.376771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:29.385657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 632/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:29.391398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.488509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e0da-6235-4806-9403-f03166d30337 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.520298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.520908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.521192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.521469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.532042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.533477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:29.547151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:29.548272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:29.549291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:29.549291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:29.549291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:29.549291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 637/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:29.553751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.649724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97920e40-cd8e-41ce-a471-b372c0ceb953 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.682721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.683308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.683588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.683860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.684486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:29.684787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.686193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.697288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 633/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:29.703180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.733834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:29.802970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce0ca1-31f2-4ac9-b1f5-08ec68c79854 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:29.842107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:29.842800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:29.843591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:29.843867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:29.850295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:29.851739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:29.862716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:29.863587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:29.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 638/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:29.869749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:29.927121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:29.968722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faf4b2-9027-4958-88e2-6123583c4e68 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:30.000774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:30.001374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:30.001655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:30.002068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:30.002736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:30.003091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:30.004601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5301-2996-472c-b049-6665380e2e59 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:30.019066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 634/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:30.026592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:30.122774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:30.131900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa7eb5-8cf5-49f6-9bab-a2a78e0f791d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:30.165137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:30.165816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:30.166101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:30.166377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] PATH_INFO: `/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/45b541633f92420e9b118c25775d646a/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:30.168244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=92831fcbcbee481387cf1e4c50cbb32e, user_id=45b541633f92420e9b118c25775d646a, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:30.169674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e38c5-c13c-4334-95ec-2c7a58ba9d1a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:30.181996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 639/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Sep 12 15:27:30 2022] PUT /identity/v3/projects/92831fcbcbee481387cf1e4c50cbb32e/users/45b541633f92420e9b118c25775d646a/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:27:32.324838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e984f0-55ad-4765-95ec-933326f15b4c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:32.325401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e984f0-55ad-4765-95ec-933326f15b4c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:32.325401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e984f0-55ad-4765-95ec-933326f15b4c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:32.326769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 635/1274] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:32.331887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:32.332305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:32.332437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:32.350632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:32.351801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:32.415477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 640/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:32.683022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e984f0-55ad-4765-95ec-933326f15b4c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:32.782846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e984f0-55ad-4765-95ec-933326f15b4c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:32.799455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:32.816147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:32.816770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:32.817055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:32.817330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:32.827996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 15:27:32.832832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:32.833959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:32.834854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 636/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Sep 12 15:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:32.839798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:32.919419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:32.937642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ebd11-7d9f-4afc-af27-81390f67e015 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:32.971027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:32.971624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:32.971905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:32.972176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:32.972829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:32.973131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:32.974514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:32.982245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 641/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Sep 12 15:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:32.988247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:33.095380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dd868-fdf7-4382-b69b-362839ac864a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:33.128565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:33.129191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:33.129474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:33.129775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:33.139428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:33.141471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:33.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:33.156674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:33.157626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:33.157626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:33.157626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:33.157626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 637/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:33.162560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:33.266925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49611de3-279e-4d24-9151-b80b9a60bb0b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:33.298912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:33.299513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:33.299799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:33.300075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:33.300709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:33.301021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:33.302416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:33.314416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 642/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:33.320111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:33.350738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:33.419957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d5313-86cf-4929-b96a-aa4ce08b733d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:33.452839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:33.453451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:33.453751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:33.454025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:33.461038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:33.462496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 15:27:33.473300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:33.474452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:33.475615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 638/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:33.480139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:33.484398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:33.577748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d524d95-d578-4b30-91c9-58acf41c9e1f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:33.611454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:33.612055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:33.612340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:33.612617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:33.613342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:33.613651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:33.615108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:33.632157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 643/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Sep 12 15:27:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:33.638612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:33.727091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:33.737307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4b2f-34d4-4fb1-9d83-12beb56f6a98 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:33.769860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:33.770454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:33.770731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:33.771002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:33.839895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:33.841271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:33.926062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:33.937473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 639/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Sep 12 15:27:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 15:27:33.944781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:34.004383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:34.041560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ad843-e7ea-4c0a-bb82-dff20effa8be None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:34.074351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:34.074964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:34.075241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:34.075510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:34.085667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:34.087127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 15:27:34.101193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:34.102343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 15:27:34.103436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 15:27:34.103436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 15:27:34.103436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:34.103436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 644/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:34.108077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:34.204940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433950c-b7ad-45d6-bd3b-bec36e31c264 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:34.237664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:34.238279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:34.238560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:34.238831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:34.239457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:34.239757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:34.241110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:34.252361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 640/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Sep 12 15:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:34.257908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:34.353213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d6d61-3681-4240-a880-3b900a2e5ed7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:34.366649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:34.387172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:34.387792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:34.388075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:34.388351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:34.394766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:34.396199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 15:27:34.407233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:34.408276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 15:27:34.413772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 645/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Sep 12 15:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:34.414384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:34.463629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:34.512534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cbc43-b858-44a7-8f6f-1244094d59f1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:34.544193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:34.544779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:34.545057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:34.545326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:34.546654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:34.546959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:34.548373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466a99-54cc-49af-b14e-82e31fd5db68 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:34.559828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 641/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Sep 12 15:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:34.566391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:34.662955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa289-0643-4a2e-9b48-5e6bd674066f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:34.695046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:34.695629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:34.695911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:34.696186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:34.696898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] RBAC: Adding query filter params (role.id=c939df765d074277be1dc85b4ec66f05, scope.project.id=92831fcbcbee481387cf1e4c50cbb32e, user.id=45b541633f92420e9b118c25775d646a) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:34.697474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:34.698924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464675f8-af4d-4115-b2e1-5039118b709d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:34.704479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 646/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Sep 12 15:27:34 2022] GET /identity/v3/role_assignments?user.id=45b541633f92420e9b118c25775d646a&scope.project.id=92831fcbcbee481387cf1e4c50cbb32e&role.id=c939df765d074277be1dc85b4ec66f05 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:36.821567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7ae9-56a8-4981-acfc-2cfb77b6c1b1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:36.822399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7ae9-56a8-4981-acfc-2cfb77b6c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:36.822496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a7ae9-56a8-4981-acfc-2cfb77b6c1b1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:36.823760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 642/1288] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:36.829360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:36.829922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:36.829922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:36.847578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:36.848750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:36.877215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:36.912787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 647/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:37.198564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a7ae9-56a8-4981-acfc-2cfb77b6c1b1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:37.298413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a7ae9-56a8-4981-acfc-2cfb77b6c1b1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:37.308732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:37.332590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:37.333208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:37.333486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:37.333786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:37.334360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:37.335726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:37.344930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:37.346307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:37.347505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 643/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:27:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 15:27:37.352069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:37.446934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9988b-aad8-4d5c-a24b-5c027bec4c03 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:37.480071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:37.480609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:37.480888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:37.481159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:37.481821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:37.482121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:37.483507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cb035-1c64-469d-bbb5-824db285f71f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:37.488120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 648/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Sep 12 15:27:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:37.493900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:37.591492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e935014-1f86-4deb-9229-8be94cfa40a5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:37.623439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:37.624528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:37.624808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:37.625081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:37.625728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:37.626031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:37.627388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f933b0-7f62-463a-997c-2ec6f9a3e813 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:37.631749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 644/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Sep 12 15:27:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:39.732894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf09f-f5d6-4541-9a7a-6f616ba7467b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:39.733949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf09f-f5d6-4541-9a7a-6f616ba7467b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:39.733949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf09f-f5d6-4541-9a7a-6f616ba7467b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:39.735229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 649/1293] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:39.740527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed23c-a6dc-4a12-8037-6f9e2bd8551b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:39.740946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed23c-a6dc-4a12-8037-6f9e2bd8551b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:39.741072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ed23c-a6dc-4a12-8037-6f9e2bd8551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:39.759665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1c9ed23c-a6dc-4a12-8037-6f9e2bd8551b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:39.760863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c9ed23c-a6dc-4a12-8037-6f9e2bd8551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:39.789270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:39.826064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 645/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:40.088623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cf09f-f5d6-4541-9a7a-6f616ba7467b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:40.118270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:40.187560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cf09f-f5d6-4541-9a7a-6f616ba7467b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:40.219659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:40.220259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:40.220541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:40.220817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:40.221457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:40.223776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36e3e1-4039-4824-8fab-245d41645c91 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:40.238109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 650/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Sep 12 15:27:40 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:27:42.389396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5839f-9c94-40a2-9311-babf4da696f1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:42.389909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5839f-9c94-40a2-9311-babf4da696f1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:42.390004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5839f-9c94-40a2-9311-babf4da696f1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:42.391345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 646/1296] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:42.396605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455f60-d238-4baf-9546-de522e2f693f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:42.396986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455f60-d238-4baf-9546-de522e2f693f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:42.397168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455f60-d238-4baf-9546-de522e2f693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:42.415369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-87455f60-d238-4baf-9546-de522e2f693f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:42.416560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-87455f60-d238-4baf-9546-de522e2f693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:42.487737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 651/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:42.755727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5839f-9c94-40a2-9311-babf4da696f1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:42.855558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5839f-9c94-40a2-9311-babf4da696f1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:42.888397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:42.889015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:42.889297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:42.890122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:42.890707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:42.892076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:42.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:42.901856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:42.902892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 647/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:27:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:42.907703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455f60-d238-4baf-9546-de522e2f693f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:42.925263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:43.003797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455f60-d238-4baf-9546-de522e2f693f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:43.043040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:43.043650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:43.043978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:43.044299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:43.044983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] RBAC: Adding query filter params (name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:43.045292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:43.046931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:43.052796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 652/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Sep 12 15:27:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:43.058945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:43.099627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:43.166737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428510a-519e-4313-9727-9877df2529c1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:43.200277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:43.200883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:43.201170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:43.201449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:43.202205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] RBAC: Adding query filter params (interface=public, service_id=389ea47c6bf84eef9ada5241b2ac69ab, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:43.202520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:43.203890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ec69-10cf-4327-820d-9c321710abbf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:43.208820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 648/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Sep 12 15:27:43 2022] GET /identity/v3/endpoints?service_id=389ea47c6bf84eef9ada5241b2ac69ab&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:43.214533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:43.293480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:43.312240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d97142-be58-4f4c-9153-658a4f64b0ad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:43.345045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:43.345637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:43.345945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:43.346218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:43.346812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:43.348170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b958-2e2d-4437-9922-acabc70a7693 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:43.354727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 653/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Sep 12 15:27:43 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:27:45.475329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45ac2-29da-45dd-8a8f-d1207e251436 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:45.475895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45ac2-29da-45dd-8a8f-d1207e251436 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:45.475895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c45ac2-29da-45dd-8a8f-d1207e251436 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:45.477224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 649/1302] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:27:45.482345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:45.482761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:45.482902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:45.500509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:27:45.501690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:27:45.565236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 654/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:27:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 15:27:45.830911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c45ac2-29da-45dd-8a8f-d1207e251436 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:45.928050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:45.934888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c45ac2-29da-45dd-8a8f-d1207e251436 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:45.968825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:45.969438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:45.969742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:45.970015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:45.970587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:45.971936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 15:27:45.980399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 15:27:45.981590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 12 15:27:45.982623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 650/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Sep 12 15:27:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 15:27:45.987398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:46.062671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:46.085994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583b1cb-0486-44a9-8c36-539f1c754134 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:46.117684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:46.118312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:46.118594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:46.118867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:46.119485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:27:46.119788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:46.121145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71b53-fb2c-47cc-9972-5ca6f40da19b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:46.126591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 655/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Sep 12 15:27:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:27:46.132349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:27:46.230293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e463030-917d-4e93-8353-96bc5529e172 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:27:46.263510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:27:46.264105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:27:46.264410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:27:46.264691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:27:46.265348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:27:46.266750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4356d40-f3fc-4631-ba23-415bc5150175 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:27:46.285652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 651/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:27:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:27:56.123731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=30 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:27:56.281369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:50.277750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184543e-ab5f-4e7b-8973-d0356a38b2c1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:50.278274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184543e-ab5f-4e7b-8973-d0356a38b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:50.278356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184543e-ab5f-4e7b-8973-d0356a38b2c1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:50.285749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add7fa-a4e5-483a-bf6a-7416de764788 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:50.286210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add7fa-a4e5-483a-bf6a-7416de764788 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:50.286292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1add7fa-a4e5-483a-bf6a-7416de764788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:50.289964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 656/1307] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:28:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:28:50.399482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a1add7fa-a4e5-483a-bf6a-7416de764788 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:28:50.401390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1add7fa-a4e5-483a-bf6a-7416de764788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:50.565732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 652/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:28:50.850533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:50.850895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:50.851093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:50.852737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 657/1309] 10.222.0.20 () {54 vars in 965 bytes} [Mon Sep 12 15:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:28:50.859550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774faed8-a03b-4a3f-b6a3-298da4e05597 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:50.859912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774faed8-a03b-4a3f-b6a3-298da4e05597 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:50.860259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774faed8-a03b-4a3f-b6a3-298da4e05597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:50.946986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-774faed8-a03b-4a3f-b6a3-298da4e05597 None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:28:50.948186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-774faed8-a03b-4a3f-b6a3-298da4e05597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:50.970967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:51.034701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 653/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Sep 12 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:28:51.043730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:28:51.045548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:51.173031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e00d14-e361-42f7-b1fe-4d7e08a44ee8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:28:51.286815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:28:51.287878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:51.288399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:51.288777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:51.290606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:51.356495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:51.432922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:28:51.434866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21030f60-1d74-4fd8-9658-0e57f9e0cf16 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:28:51.456751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 658/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:28:53.983154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:53.983708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:53.983708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:53.986049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 654/1312] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:28:53.991660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75b58c-0030-4682-b91a-d8975703c70d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:53.992115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75b58c-0030-4682-b91a-d8975703c70d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:53.992234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75b58c-0030-4682-b91a-d8975703c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:54.071864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0c75b58c-0030-4682-b91a-d8975703c70d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:28:54.073075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0c75b58c-0030-4682-b91a-d8975703c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:54.166854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 659/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:28:54.445461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:28:54.447044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:54.549566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc18a6-6c75-4de9-9996-2e50ac45afaa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:28:54.619813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:54.622411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:28:54.622967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:54.623463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:54.623764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:54.625765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:54.733953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:28:54.735389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1face6-6543-40d6-a942-50b84f1a319e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:28:54.752843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 655/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:28:57.195084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:57.195662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:57.195662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:57.197022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 660/1315] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:28:57.202393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1cc6-6ed0-4c5f-a01f-60f84355bca2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:57.202808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1cc6-6ed0-4c5f-a01f-60f84355bca2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:57.202948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1cc6-6ed0-4c5f-a01f-60f84355bca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:57.268573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-802f1cc6-6ed0-4c5f-a01f-60f84355bca2 None None] MFA Rules not processed for user `fe943a904f9347cdbc2ea3c7493a9421`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:28:57.269823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-802f1cc6-6ed0-4c5f-a01f-60f84355bca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:57.321445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:57.366849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 656/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:28:57.657578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:28:57.659149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:57.668795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:57.760209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357b2d-1d8c-4eac-92ce-2dff99e5b84e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:28:57.832489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:28:57.833080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:28:57.833363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:28:57.833628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:28:57.835222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:28:57.896630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:28:57.942291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:28:57.943722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ab2e-a024-447a-8a3b-5116610e8144 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:28:57.962152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 661/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:28:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:00.611814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:00.612373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:00.612373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:00.613758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 657/1318] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:00.618837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da949d-2c74-407c-a3fc-f94462a59b61 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:00.619256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da949d-2c74-407c-a3fc-f94462a59b61 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:00.619404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da949d-2c74-407c-a3fc-f94462a59b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:00.684918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-29da949d-2c74-407c-a3fc-f94462a59b61 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:00.686641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-29da949d-2c74-407c-a3fc-f94462a59b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:00.778803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 662/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:01.066048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:01.067624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:01.171247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d99c5-1ce2-42ae-9b1e-83750772d6c4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:01.243999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:01.244611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:01.244894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:01.245169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:01.246821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:01.254459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:01.355605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:01.356961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5920-98a9-4132-983e-a655e8135771 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:01.376066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 658/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:04.146405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:04.146973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:04.146973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:04.148484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 663/1321] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:04.153691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f0b7-e300-4024-b954-f2481a3b4ecc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:04.154144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f0b7-e300-4024-b954-f2481a3b4ecc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:04.154322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693f0b7-e300-4024-b954-f2481a3b4ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:04.227230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c693f0b7-e300-4024-b954-f2481a3b4ecc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:04.228472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c693f0b7-e300-4024-b954-f2481a3b4ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:04.293136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:04.336790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 659/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:04.616858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:04.618715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:04.638473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:04.723170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253be48-d895-49f0-ae91-5b1d7f1151e0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:04.796286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:04.796889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:04.797172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:04.797444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:04.799083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:04.870298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:04.909483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:04.910970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5c2ee-cecd-46a8-973d-ebcca23dcef7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:04.929651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 664/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:07.150483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:07.151016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:07.151016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:07.152393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 660/1324] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:07.158129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a8d24-d2e0-4226-8861-afc2ce7492e4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:07.158568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a8d24-d2e0-4226-8861-afc2ce7492e4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:07.158811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a8d24-d2e0-4226-8861-afc2ce7492e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:07.225404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-dd8a8d24-d2e0-4226-8861-afc2ce7492e4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:07.226725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dd8a8d24-d2e0-4226-8861-afc2ce7492e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:07.333302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 665/1325] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:07.612726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:07.614384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:07.721493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853708d-8675-4f8f-af2d-0d6f68cbb5df None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:07.795622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:07.796248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:07.796530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:07.796797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:07.798416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:07.815672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:07.907501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:07.908935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade4e07-ee2b-4959-bf12-2d0c2515183f service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:07.927068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 661/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:10.123885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:10.124390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:10.124390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:10.125729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 666/1327] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:10.130908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e864c4-ecf3-474c-80ef-429f77e0afca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:10.131339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e864c4-ecf3-474c-80ef-429f77e0afca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:10.131466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e864c4-ecf3-474c-80ef-429f77e0afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:10.197497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d8e864c4-ecf3-474c-80ef-429f77e0afca None None] MFA Rules not processed for user `fe943a904f9347cdbc2ea3c7493a9421`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:10.198750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d8e864c4-ecf3-474c-80ef-429f77e0afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:10.266635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:10.298874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 662/1328] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:10.594697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:10.596249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:10.629213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:10.706377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a0ec6-c5cf-41e5-8faf-0d2d7f3180e6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:10.781215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:10.781863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:10.782148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:10.782420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:10.784024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:10.861246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:10.895094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:10.896533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d3da7-4143-446f-8ec4-e6a898828544 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:10.918999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 667/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:13.292950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:13.293494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:13.293494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:13.294919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 663/1330] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:13.300021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d790ec-b0dd-40a7-b354-9b9c47f34149 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:13.300405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d790ec-b0dd-40a7-b354-9b9c47f34149 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:13.300597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d790ec-b0dd-40a7-b354-9b9c47f34149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:13.364590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b2d790ec-b0dd-40a7-b354-9b9c47f34149 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:13.365829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b2d790ec-b0dd-40a7-b354-9b9c47f34149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:13.459006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 668/1331] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:13.736024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:13.737568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:13.841315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120120-80be-473f-844b-bde4c66f0d7b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:13.914577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:13.915179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:13.915463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:13.915734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:13.917320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:13.944719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:14.027934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:14.029371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa5fa-cc53-4e29-bfee-8659aa1b6a0b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:14.047308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 664/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:16.216055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:16.216632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:16.216632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:16.218052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 669/1333] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:16.223942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e869c69-5f43-464e-b143-0d0e61b43e4d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:16.224357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e869c69-5f43-464e-b143-0d0e61b43e4d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:16.224500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e869c69-5f43-464e-b143-0d0e61b43e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:16.291175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7e869c69-5f43-464e-b143-0d0e61b43e4d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:16.292395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e869c69-5f43-464e-b143-0d0e61b43e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:16.369519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:16.392530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 665/1334] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:16.674789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:16.676317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:16.716684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:16.780069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9c308-1fae-468c-83cb-95835341cb36 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:16.851098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:16.851692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:16.851970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:16.852236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:16.853830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:16.936487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:16.962339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:16.963785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5ba60-4c03-4551-aaac-87c399c07b13 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:16.981599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 670/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:19.457991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:19.458533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:19.458533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:19.459860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 666/1336] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:19.465053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9be0-d4b1-4bff-8b56-2b5be69dab81 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:19.465478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9be0-d4b1-4bff-8b56-2b5be69dab81 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:19.465633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f9be0-d4b1-4bff-8b56-2b5be69dab81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:19.530722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-333f9be0-d4b1-4bff-8b56-2b5be69dab81 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:19.531933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-333f9be0-d4b1-4bff-8b56-2b5be69dab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:19.625837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 671/1337] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:19.903465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:19.904996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:20.007631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a72b46-c974-48c3-a2b4-ce9a0f01e757 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:20.082137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:20.082743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:20.083022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:20.083293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:20.084872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:20.120155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:20.191534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:20.192900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e454bd4-e9bd-43af-a365-7e04bcd4e9f8 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:20.211439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 667/1338] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:24.345240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:24.345831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:24.345831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:24.347081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 672/1339] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:24.352346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1bcce-d473-41b5-8c28-1bcf334805b3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:24.352773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1bcce-d473-41b5-8c28-1bcf334805b3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:24.352905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1bcce-d473-41b5-8c28-1bcf334805b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:24.419265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d7a1bcce-d473-41b5-8c28-1bcf334805b3 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:24.421001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d7a1bcce-d473-41b5-8c28-1bcf334805b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:24.508713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:24.521879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 668/1340] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:24.800664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:24.802254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:24.852074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:24.906944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca60f3-e7b0-4cf7-b922-c7fe0bd11663 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:24.980070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:24.980677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:24.980962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:24.981240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:24.983002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:25.077381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:25.094068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:25.095515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efaa38-e9e6-4f17-90ac-dfe787987a80 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:25.113808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 673/1341] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:27.913237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:27.913822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:27.913936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:27.915358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 669/1342] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:27.920564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984ae1-022b-4321-90b1-99060ae1b0d3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:27.920999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984ae1-022b-4321-90b1-99060ae1b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:27.921178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984ae1-022b-4321-90b1-99060ae1b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:27.986613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-51984ae1-022b-4321-90b1-99060ae1b0d3 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:27.987968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51984ae1-022b-4321-90b1-99060ae1b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:28.085576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 674/1343] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:28.364416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:28.366215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:28.479660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd7bca-58fa-4d52-82ac-dd923f51b78c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:28.555680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:28.556204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:28.556488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:28.556759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:28.558406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:28.603281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:28.671725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:28.673173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad7437-cbdd-4628-b398-5a36a290548a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:28.691736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 670/1344] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:32.820131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:32.820665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:32.820665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:32.822122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 675/1345] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:32.827395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f639-1627-4776-a7a6-1f4da89f5e76 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:32.827777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f639-1627-4776-a7a6-1f4da89f5e76 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:32.827950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f639-1627-4776-a7a6-1f4da89f5e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:32.899478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6743f639-1627-4776-a7a6-1f4da89f5e76 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:32.900709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6743f639-1627-4776-a7a6-1f4da89f5e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:33.000595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:33.002996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 671/1346] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:33.286624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:33.288173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:33.345114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:33.390730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59b32e-06b4-44dc-98ac-0a2e23c3f4a7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:33.463066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:33.463666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:33.463948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:33.464220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:33.465828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:33.577201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:33.581644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:33.583156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f838b46-9d99-472c-ba0d-e425d188c6f4 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:33.603731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 676/1347] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:38.511210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ef1b-dc08-4317-a25f-4284c6695cd9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:38.511663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ef1b-dc08-4317-a25f-4284c6695cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:38.511784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ef1b-dc08-4317-a25f-4284c6695cd9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:38.515934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 672/1348] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:29:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:38.522033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad285-dcf7-4c4b-a196-65ffa4edef13 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:38.522033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad285-dcf7-4c4b-a196-65ffa4edef13 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:38.522601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ad285-dcf7-4c4b-a196-65ffa4edef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:38.632781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fe5ad285-dcf7-4c4b-a196-65ffa4edef13 None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:38.634472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fe5ad285-dcf7-4c4b-a196-65ffa4edef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:38.811707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 677/1349] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:29:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:42.954488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0961-19a1-4113-b430-3bce6f125ef5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:42.955056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0961-19a1-4113-b430-3bce6f125ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:42.955056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea0961-19a1-4113-b430-3bce6f125ef5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:42.956460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 673/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:42.961777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f02-31be-473a-911d-53fb6547603e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:42.962189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f02-31be-473a-911d-53fb6547603e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:42.962342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf85f02-31be-473a-911d-53fb6547603e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:43.021611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:43.042922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4bf85f02-31be-473a-911d-53fb6547603e None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:43.044274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4bf85f02-31be-473a-911d-53fb6547603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:43.153755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 678/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:44.605405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625443d9-18ea-449d-866b-efe06f2fab8a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:44.605985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625443d9-18ea-449d-866b-efe06f2fab8a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:44.606110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625443d9-18ea-449d-866b-efe06f2fab8a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:44.608177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 674/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:44.613319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703db08-d501-4d4b-bff3-e9bc1000e3af None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:44.613807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703db08-d501-4d4b-bff3-e9bc1000e3af None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:44.613995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703db08-d501-4d4b-bff3-e9bc1000e3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:44.691340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b703db08-d501-4d4b-bff3-e9bc1000e3af None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:44.692511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b703db08-d501-4d4b-bff3-e9bc1000e3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:44.734424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:44.799622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 679/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:29:48.312579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c883-cc51-4849-91fd-d88fd5be381d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:48.313457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c883-cc51-4849-91fd-d88fd5be381d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:48.313782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c883-cc51-4849-91fd-d88fd5be381d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:48.315174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 675/1354] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:48.320814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d6ac-bdf8-43f5-8871-efe2e03f0e1b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:48.321257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d6ac-bdf8-43f5-8871-efe2e03f0e1b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:48.321371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d6ac-bdf8-43f5-8871-efe2e03f0e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:48.398236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1f70d6ac-bdf8-43f5-8871-efe2e03f0e1b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:48.399451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1f70d6ac-bdf8-43f5-8871-efe2e03f0e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:48.502649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 680/1355] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:48.804987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:48.805541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:48.806002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:48.807533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 676/1356] 10.222.0.20 () {54 vars in 962 bytes} [Mon Sep 12 15:29:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:48.816875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c0d76-6ad2-44fa-8689-09f9c30b30ed None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:48.817841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c0d76-6ad2-44fa-8689-09f9c30b30ed None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:48.818400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c0d76-6ad2-44fa-8689-09f9c30b30ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:48.842458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:48.905604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-324c0d76-6ad2-44fa-8689-09f9c30b30ed None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:48.906863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-324c0d76-6ad2-44fa-8689-09f9c30b30ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:49.030601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 681/1357] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Sep 12 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:29:49.038326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:49.040014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:49.213208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f61b9-b03a-44ae-bd71-0a84d76b328e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:49.293265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:49.344522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:49.345262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:49.345625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:49.346003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:49.347718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:49.458438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:49.460061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ddfd6-cb8e-418b-ba4c-0086f1f0e13d service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:49.479976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 677/1358] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:51.844085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:51.844577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:51.844577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:51.846394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 682/1359] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:51.854263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e1e-4dd2-4744-b28d-213ec06ac23d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:51.854263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e1e-4dd2-4744-b28d-213ec06ac23d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:51.854263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e1e-4dd2-4744-b28d-213ec06ac23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:51.944611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6d878e1e-4dd2-4744-b28d-213ec06ac23d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:51.946024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6d878e1e-4dd2-4744-b28d-213ec06ac23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:51.987316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:52.059815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 678/1360] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:52.347272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:52.348795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:52.439676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:52.545911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ea41c-e3d6-4e6a-8d86-73bc81d89429 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:52.695975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:52.696618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:52.696903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:52.697177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:52.698860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:52.730102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:52.801332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:52.802996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c967a-49b3-46fe-9690-c2f7c412b325 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:52.821244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 683/1361] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:29:55.310039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9827fb-7360-44d1-93e1-f64747e6dc6c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:55.310608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9827fb-7360-44d1-93e1-f64747e6dc6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:55.310608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9827fb-7360-44d1-93e1-f64747e6dc6c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:55.312114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 679/1362] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:55.317460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95998-fd21-446a-bd7f-ded4b4d65e48 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:55.317910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95998-fd21-446a-bd7f-ded4b4d65e48 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:55.318162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95998-fd21-446a-bd7f-ded4b4d65e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:55.394070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-eac95998-fd21-446a-bd7f-ded4b4d65e48 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:55.395433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eac95998-fd21-446a-bd7f-ded4b4d65e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:55.503675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:55.511865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 684/1363] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:55.793976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:55.794390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:55.794778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:55.796265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 680/1364] 10.222.0.20 () {54 vars in 962 bytes} [Mon Sep 12 15:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:55.801652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049dac-1a4a-406f-8a3f-39c817c215d7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:55.801986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049dac-1a4a-406f-8a3f-39c817c215d7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:55.802268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c049dac-1a4a-406f-8a3f-39c817c215d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:55.882249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8c049dac-1a4a-406f-8a3f-39c817c215d7 None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:55.883798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8c049dac-1a4a-406f-8a3f-39c817c215d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:56.006340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 685/1365] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Sep 12 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:56.014717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:56.016267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:56.172174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a1c5-4722-4cbe-b461-0a2563645dbd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:56.194410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:56.304545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:56.305168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:56.305452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:56.305751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:56.307634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:56.420144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:56.422467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8388a-7a9f-428b-94c0-80d0fc7940d4 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:56.445274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 681/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:29:56.520979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08981df3-1e27-406c-b662-d15224dc4984 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:56.521491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08981df3-1e27-406c-b662-d15224dc4984 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:56.521602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08981df3-1e27-406c-b662-d15224dc4984 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:56.522894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 686/1367] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:56.528227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290614-ffc1-4d9b-a396-0614ada68fca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:56.528515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290614-ffc1-4d9b-a396-0614ada68fca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:56.529333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290614-ffc1-4d9b-a396-0614ada68fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:56.564366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:56.608355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d2290614-ffc1-4d9b-a396-0614ada68fca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:56.609594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d2290614-ffc1-4d9b-a396-0614ada68fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:56.735907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 682/1368] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:29:57.039298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:57.039587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:57.039861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:57.041336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 687/1369] 10.222.0.20 () {54 vars in 964 bytes} [Mon Sep 12 15:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:57.046790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754150-1f8c-4c69-83bf-9711619209cd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:57.047090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754150-1f8c-4c69-83bf-9711619209cd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:57.047368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29754150-1f8c-4c69-83bf-9711619209cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:57.126213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-29754150-1f8c-4c69-83bf-9711619209cd None None] MFA Rules not processed for user `44a33c23a10f4ae5a008e1d4d4973355`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:57.127423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-29754150-1f8c-4c69-83bf-9711619209cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:57.178671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:57.213408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 683/1370] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Sep 12 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:57.221605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:57.223233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:57.276620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:57.334463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c5176-cc05-42da-9d41-18ec4ee15976 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:57.424653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:57.425426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:57.425838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:57.426206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:57.428054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:57.542696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:57.558629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:57.560314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c1f28-71a1-4f80-8186-da098a865600 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:57.582126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 688/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:29:58.689090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:58.689682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:58.689682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:58.691067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 684/1372] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:29:58.696440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962ed-c9e1-4114-9491-4d1838f5347d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:58.696873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962ed-c9e1-4114-9491-4d1838f5347d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:58.696997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f962ed-c9e1-4114-9491-4d1838f5347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:58.766894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-39f962ed-c9e1-4114-9491-4d1838f5347d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:29:58.768117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-39f962ed-c9e1-4114-9491-4d1838f5347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:58.863943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 689/1373] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:29:59.161892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:29:59.163539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:59.318127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeafba-6f6d-4df0-8dee-c343e0c5302b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:29:59.376283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:29:59.442695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:29:59.443320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:29:59.443601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:29:59.443876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:29:59.445482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:29:59.540858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:29:59.542539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249ebe2-ef16-4009-acd7-a349adcc85f4 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:29:59.561577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 685/1374] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:01.786460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:01.786988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:01.786988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:01.789750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 690/1375] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:01.793920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9fe5-a86c-41ed-8ff2-66b7f2198f0d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:01.794369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9fe5-a86c-41ed-8ff2-66b7f2198f0d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:01.795170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a9fe5-a86c-41ed-8ff2-66b7f2198f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:01.866143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e89a9fe5-a86c-41ed-8ff2-66b7f2198f0d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:01.867494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e89a9fe5-a86c-41ed-8ff2-66b7f2198f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:01.875172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:01.981838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 686/1376] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:02.271431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:02.273116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:02.337295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:02.432861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b561ac-a337-4688-92fe-397b7a1e9eca None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:02.557747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:02.558360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:02.558658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:02.558932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:02.561068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:02.581410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:02.658402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:02.660033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298fa7d-7b57-489d-b183-39c76c1f414a service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:02.681625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 691/1377] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:04.907799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:04.908292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:04.908378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:04.913633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 687/1378] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:04.915350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9b4c-bbfd-4634-9e8f-bef3ccbc04f8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:04.915788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9b4c-bbfd-4634-9e8f-bef3ccbc04f8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:04.915980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f9b4c-bbfd-4634-9e8f-bef3ccbc04f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:04.989354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e54f9b4c-bbfd-4634-9e8f-bef3ccbc04f8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:04.990471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e54f9b4c-bbfd-4634-9e8f-bef3ccbc04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:05.088016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:05.098519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 692/1379] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:05.429957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:05.432219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:05.594648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:05.605463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c4bb9-17cc-40f9-bc33-a23811ea9ac9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:05.725036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:05.725569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:05.725893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:05.726168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:05.727782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:05.824821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:05.828257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba366-1231-4c66-b7ba-0e365042848c service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:05.843739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:05.852722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 688/1380] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:08.068820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:08.069351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:08.069351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:08.070850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 693/1381] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:08.076778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d789-f59d-4950-a79c-913c431417a2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:08.077423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d789-f59d-4950-a79c-913c431417a2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:08.077423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8d789-f59d-4950-a79c-913c431417a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:08.147893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bea8d789-f59d-4950-a79c-913c431417a2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:08.149118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bea8d789-f59d-4950-a79c-913c431417a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:08.265853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 689/1382] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:08.649137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:08.650657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:08.802542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d687a6-312d-4eed-b95f-c85bedf743b9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:08.878815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:08.932081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:08.932753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:08.933137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:08.933547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:08.936276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:09.056356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:09.058040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20045024-57f1-4beb-87eb-369b66060efc service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:09.081797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 694/1383] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:30:11.264264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:11.264264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:11.265114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:11.265775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 690/1384] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:11.271053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8df49-86e8-4791-905d-f5f812c88185 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:11.271483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8df49-86e8-4791-905d-f5f812c88185 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:11.271604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8df49-86e8-4791-905d-f5f812c88185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:11.338057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-dea8df49-86e8-4791-905d-f5f812c88185 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:11.339277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dea8df49-86e8-4791-905d-f5f812c88185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:11.377316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:11.442243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 695/1385] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:30:11.730800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:11.732939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:11.814262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:11.898212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33929fcc-8c6d-4077-af4f-f2dfd95855cd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:12.026174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:12.026888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:12.027194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:12.027450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:12.029173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:12.078586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:12.135719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:12.137347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ad8eb-5833-4a8f-974d-5c3d66b6925a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:12.155963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 691/1386] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:12.848168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e38f7-6241-407e-93df-1744a7e31fc0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:12.848995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e38f7-6241-407e-93df-1744a7e31fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:12.848995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e38f7-6241-407e-93df-1744a7e31fc0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:12.856641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9b94-7c93-47cb-87c4-10368d46f26d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:12.857186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9b94-7c93-47cb-87c4-10368d46f26d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:12.857272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9b94-7c93-47cb-87c4-10368d46f26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:12.863660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 696/1387] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:12 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:12.959269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-230f9b94-7c93-47cb-87c4-10368d46f26d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:12.960441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-230f9b94-7c93-47cb-87c4-10368d46f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:13.081034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 692/1388] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:13.384386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:13.384678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:13.384961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:13.386339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 697/1389] 10.222.0.20 () {54 vars in 966 bytes} [Mon Sep 12 15:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:13.392366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a6c1-eaa3-436d-a26e-0f675b2d6793 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:13.392843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a6c1-eaa3-436d-a26e-0f675b2d6793 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:13.393060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a6c1-eaa3-436d-a26e-0f675b2d6793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:13.410850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:13.474370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0284a6c1-eaa3-436d-a26e-0f675b2d6793 None None] MFA Rules not processed for user `9ae494b13b5743a2b56054d2fae338fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:13.475519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0284a6c1-eaa3-436d-a26e-0f675b2d6793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:13.553909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 693/1390] 10.222.0.20 () {58 vars in 1063 bytes} [Mon Sep 12 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:13.565675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:13.567336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:13.672984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91659-ef5a-444f-9884-1122b23b042d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:13.734590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:13.750295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:13.750907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:13.751191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:13.751468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:13.753072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:13.863888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:13.866055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c95ca-13b1-42e6-864d-cea246fc78d2 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:13.886192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 698/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:14.382793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717a3a0-ff41-42f0-809a-b7ed4dd10cf5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:14.383236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717a3a0-ff41-42f0-809a-b7ed4dd10cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:14.383344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717a3a0-ff41-42f0-809a-b7ed4dd10cf5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:14.384679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 694/1392] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:14.389991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d5c2b-e794-48c4-a091-89d6e5b40632 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:14.390285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d5c2b-e794-48c4-a091-89d6e5b40632 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:14.390559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d5c2b-e794-48c4-a091-89d6e5b40632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:14.457043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e37d5c2b-e794-48c4-a091-89d6e5b40632 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:14.458319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e37d5c2b-e794-48c4-a091-89d6e5b40632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:14.505887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:14.566785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 699/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:14.643080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb057-c815-4398-81c8-b9b2a58044b8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:14.643398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb057-c815-4398-81c8-b9b2a58044b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:14.643680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb057-c815-4398-81c8-b9b2a58044b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:14.712918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6c3eb057-c815-4398-81c8-b9b2a58044b8 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:14.714178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c3eb057-c815-4398-81c8-b9b2a58044b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:14.769242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:14.803005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 695/1394] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:14.845286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:14.845642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:14.845950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:14.847404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 700/1395] 10.222.0.20 () {54 vars in 943 bytes} [Mon Sep 12 15:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:14.854051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:14.854344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:14.854749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:14.856181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:14.857785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:14.937241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] MFA Rules not processed for user `c9679f016e524268b81488141bfc8777`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:14.938652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:15.006984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0857-a23a-4164-bbed-96a2c1737aaf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:15.034286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:15.049507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 696/1396] 10.222.0.20 () {58 vars in 1040 bytes} [Mon Sep 12 15:30:14 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:15.058510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:15.068427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ad0f-8718-4481-bf78-c09666a343bb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:15.096008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:15.125315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fc502-7a99-4791-9453-166459496b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:15.126046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc502-7a99-4791-9453-166459496b32 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:15.126404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc502-7a99-4791-9453-166459496b32 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:15.126685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc502-7a99-4791-9453-166459496b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:15.128286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-085fc502-7a99-4791-9453-166459496b32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:15.179351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:15.179954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:15.180236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:15.180510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:15.182168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:15.220594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc502-7a99-4791-9453-166459496b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:15.222241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc502-7a99-4791-9453-166459496b32 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:15.241224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 701/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:30:15.270773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:15.272545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801eb8-36f5-46ae-b31b-8cc9e0b21ab0 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:15.281008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 697/1398] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 15:30:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 15:30:17.467608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:17.468144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:17.468144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:17.469505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 702/1399] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:17.477094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6721a-b32f-4995-95e0-e0dc4a98d692 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:17.478138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6721a-b32f-4995-95e0-e0dc4a98d692 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:17.478860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6721a-b32f-4995-95e0-e0dc4a98d692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:17.585530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-72d6721a-b32f-4995-95e0-e0dc4a98d692 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:17.587082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-72d6721a-b32f-4995-95e0-e0dc4a98d692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:17.598388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:17.718835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 698/1400] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:18.010224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:18.012038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:18.048964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:18.175360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9dc8e-b618-447c-a192-3f5572b3fedb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:18.292133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:18.303970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:18.304670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:18.305018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:18.305359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:18.307030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:18.416099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:18.417857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca2c51-777c-4e95-a23f-4376888c4d67 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:18.436864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 703/1401] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:20.676326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:20.676933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:20.676933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:20.678509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 699/1402] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:20.683835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5e28-3cdd-4687-9955-dd9db7bdbd45 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:20.684349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5e28-3cdd-4687-9955-dd9db7bdbd45 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:20.684970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae5e28-3cdd-4687-9955-dd9db7bdbd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:20.758250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e1ae5e28-3cdd-4687-9955-dd9db7bdbd45 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:20.759505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e1ae5e28-3cdd-4687-9955-dd9db7bdbd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:20.827471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:20.874104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 704/1403] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:21.170375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:21.171914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:21.322980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:21.334183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe9694-5b4f-422b-bbef-b67050b46440 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:21.469118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:21.469795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:21.470156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:21.470441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:21.472066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:21.583007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:21.584776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41050de1-1af6-47ba-8b30-ff43af8fd0b4 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:21.596264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:21.606030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 700/1404] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:23.838269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:23.838269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:23.838938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:23.845008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 705/1405] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:30:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:23.846132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d070e8-7a22-47d3-8422-63259dd1aa77 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:23.846572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d070e8-7a22-47d3-8422-63259dd1aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:23.846676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d070e8-7a22-47d3-8422-63259dd1aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:23.919032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b8d070e8-7a22-47d3-8422-63259dd1aa77 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:23.920275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8d070e8-7a22-47d3-8422-63259dd1aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:24.036578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 701/1406] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:24.354272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:24.355991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:24.526288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bdcfc-6028-4f8f-b1f0-fd2fe8a7bb00 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:24.589093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:24.663415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:24.664117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:24.664405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:24.664681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:24.666326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:24.787292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:24.788905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe48a8-a517-4c1e-9e48-1b1736143c40 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:24.811741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 706/1407] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:30:34.038055 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=67 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:34.808769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=76 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:38.568315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db296a-e082-46fb-b006-e841ac33952f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:38.568898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db296a-e082-46fb-b006-e841ac33952f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:38.568898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db296a-e082-46fb-b006-e841ac33952f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:38.570238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 702/1408] 10.222.0.20 () {54 vars in 918 bytes} [Mon Sep 12 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:38.576028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007312db-8e7f-488a-b2d4-3773adcc3028 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:38.576466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007312db-8e7f-488a-b2d4-3773adcc3028 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:38.576585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007312db-8e7f-488a-b2d4-3773adcc3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:38.650230 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-007312db-8e7f-488a-b2d4-3773adcc3028 None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:38.651381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-007312db-8e7f-488a-b2d4-3773adcc3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:38.758260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 707/1409] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Sep 12 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:38.772666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4563dc-a2a0-40cb-b145-00c66584c883 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:38.772953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4563dc-a2a0-40cb-b145-00c66584c883 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:38.773255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4563dc-a2a0-40cb-b145-00c66584c883 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:38.774646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 703/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Sep 12 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:38.779124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4677fd-7005-459e-a99f-8287b90b0d2b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:38.779398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4677fd-7005-459e-a99f-8287b90b0d2b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:38.779675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4677fd-7005-459e-a99f-8287b90b0d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:38.851845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:38.854684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3b4677fd-7005-459e-a99f-8287b90b0d2b None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:38.856968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b4677fd-7005-459e-a99f-8287b90b0d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:38.975625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 708/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Sep 12 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:41.586113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:41.586569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:41.586680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:41.588016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 704/1412] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:41.593497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc18128-e56e-4188-b775-e199cc42cbf5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:41.593941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc18128-e56e-4188-b775-e199cc42cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:41.594097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc18128-e56e-4188-b775-e199cc42cbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:41.659869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cfc18128-e56e-4188-b775-e199cc42cbf5 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:41.661024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cfc18128-e56e-4188-b775-e199cc42cbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:41.721618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:41.763657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 709/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:42.063504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:42.065120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:42.211331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95a734-7ff5-4e4d-8dbc-1795961fc3ee None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:42.283562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:42.323684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:42.324225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:42.324766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:42.325042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:42.326674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:42.415739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:42.418005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645d38-a3f1-4666-b0ce-ab7ac7d648f4 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:42.435670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 705/1414] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:43.562521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:43.563096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:43.563186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:43.564494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 710/1415] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:43.570639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd2d1-4846-46bb-a735-a57148095e5d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:43.571029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd2d1-4846-46bb-a735-a57148095e5d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:43.571131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fd2d1-4846-46bb-a735-a57148095e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:43.648631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b66fd2d1-4846-46bb-a735-a57148095e5d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:43.649798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b66fd2d1-4846-46bb-a735-a57148095e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:43.677456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:43.748630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 706/1416] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:30:43.899982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:43.901503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:44.046550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32819e76-81cc-44cd-931b-ffc9024da1a2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:44.086818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:44.158092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:44.158624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:44.158907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:44.159181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:44.160767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:44.251047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:44.252657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079332cd-cccb-4164-a51f-42c35933e25c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:44.270422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 711/1417] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:45.400615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:45.401058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:45.401165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:45.402584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 707/1418] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:45.409014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e8fe-befa-40fd-aeb5-580792bc05c0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:45.409457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e8fe-befa-40fd-aeb5-580792bc05c0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:45.409577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595e8fe-befa-40fd-aeb5-580792bc05c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:45.463418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:45.486644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4595e8fe-befa-40fd-aeb5-580792bc05c0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:45.487779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4595e8fe-befa-40fd-aeb5-580792bc05c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:45.582576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 712/1419] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:45.734090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:45.735603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:45.880526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2fb51-e02a-43ba-80c3-e070091a4e92 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:45.900986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:46.001717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:46.002318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:46.002595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:46.002864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:46.004468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:46.092809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:46.094451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc986d4e-78eb-454d-9827-9864f202ec80 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:46.112231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 708/1420] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:47.219382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:47.219981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:47.219981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:47.221285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 713/1421] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:47.227561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c7c0-0333-4635-88c3-8a419946c099 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:47.227974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c7c0-0333-4635-88c3-8a419946c099 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:47.228106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45c7c0-0333-4635-88c3-8a419946c099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:47.247483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:47.303685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2b45c7c0-0333-4635-88c3-8a419946c099 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:47.304895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2b45c7c0-0333-4635-88c3-8a419946c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:47.400985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 709/1422] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:47.547745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:47.549299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:47.657430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:47.693353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48186cd1-f359-4649-8d89-af01cf59eb36 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:47.803093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:47.803621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:47.803902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:47.804174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:47.805784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:47.879782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:47.893433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:47.895070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1faa6-a7bd-45c5-becf-137f5430601c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:47.912541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 714/1423] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:49.022858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:49.023430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:49.023430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:49.024802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 710/1424] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:49.031445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b4fd-63fd-41e4-8657-08c9460193f6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:49.031862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b4fd-63fd-41e4-8657-08c9460193f6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:49.032004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b4fd-63fd-41e4-8657-08c9460193f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:49.111412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3449b4fd-63fd-41e4-8657-08c9460193f6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:49.112637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3449b4fd-63fd-41e4-8657-08c9460193f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:49.211847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 715/1425] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:49.363101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:49.364586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:49.449552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:49.509885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8649-4781-4777-9442-428d6f47f6c3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:49.622572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:49.623118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:49.623402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:49.623680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:49.625276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:49.673276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:49.715246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:49.716859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5801e-3b4e-4076-a618-8ed75412ac3e service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:49.735265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 711/1426] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:50.801676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:50.802238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:50.802238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:50.803611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 716/1427] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:50.809919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64869bc-d5c0-4d6d-a5a3-c1bc7373716e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:50.810344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64869bc-d5c0-4d6d-a5a3-c1bc7373716e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:50.810477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64869bc-d5c0-4d6d-a5a3-c1bc7373716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:50.886940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f64869bc-d5c0-4d6d-a5a3-c1bc7373716e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:50.888157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f64869bc-d5c0-4d6d-a5a3-c1bc7373716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:50.992608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 712/1428] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:51.144977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:51.146537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:51.194630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:51.299787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf6970-b641-4871-acc4-eaf3b27e42ab None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:51.414932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:51.415543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:51.415825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:51.416104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:51.418255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:51.425195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:51.508002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:51.509607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22e234-2404-4570-b645-0a1e6672025c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:51.527588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 717/1429] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:52.738160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:52.738693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:52.738693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:52.739936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 713/1430] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:52.746741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5c49-2ed5-46da-a1f6-7daf722beea1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:52.747226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5c49-2ed5-46da-a1f6-7daf722beea1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:52.747342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5c49-2ed5-46da-a1f6-7daf722beea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:52.823846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7c9d5c49-2ed5-46da-a1f6-7daf722beea1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:52.825325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7c9d5c49-2ed5-46da-a1f6-7daf722beea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:52.891090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:52.919455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 718/1431] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:53.075066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:53.076577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:53.094898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:53.223775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e4f9-f8fc-4d78-9e8c-1cc4da5459ea None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:53.315956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:53.343387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395148-0198-441e-926c-248b56585934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:53.343924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395148-0198-441e-926c-248b56585934 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:53.344210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395148-0198-441e-926c-248b56585934 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:53.344485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395148-0198-441e-926c-248b56585934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:53.346126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-23395148-0198-441e-926c-248b56585934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:53.436616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23395148-0198-441e-926c-248b56585934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:53.438325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23395148-0198-441e-926c-248b56585934 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:53.456626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 714/1432] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:54.516725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:54.517293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:54.517293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:54.518721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 719/1433] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:54.525183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a21a8-cf71-4339-a653-7ba1c1de7f86 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:54.525644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a21a8-cf71-4339-a653-7ba1c1de7f86 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:54.526337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a21a8-cf71-4339-a653-7ba1c1de7f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:54.602653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-082a21a8-cf71-4339-a653-7ba1c1de7f86 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:54.603796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-082a21a8-cf71-4339-a653-7ba1c1de7f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:54.647160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:54.702509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 715/1434] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:54.853476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:54.855051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:54.999632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82ecad-34b2-46dd-87f1-8c6a7a922f0b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:55.053358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:55.117133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:55.117669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:55.117984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:55.118261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:55.120365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:55.208802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:55.210464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3ae7-f024-4a9d-bea5-72d993a12b01 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:55.228165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 720/1435] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:56.295080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:56.295660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:56.295660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:56.297135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 716/1436] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:56.303531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671a1fe-059a-41f5-b401-f4db5e685ebe None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:56.304026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671a1fe-059a-41f5-b401-f4db5e685ebe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:56.304124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671a1fe-059a-41f5-b401-f4db5e685ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:56.381642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e671a1fe-059a-41f5-b401-f4db5e685ebe None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:56.382950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e671a1fe-059a-41f5-b401-f4db5e685ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:56.384984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:56.478338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 721/1437] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:56.625241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:56.626724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:56.770789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5f090-70a2-4152-bfd3-2fd550eadca1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:56.797540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:56.883394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:56.883924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:56.884207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:56.884481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:56.886130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:56.974225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:56.975811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d3cb-1ab0-44be-b4d3-2f23418bf654 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:56.993817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 717/1438] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:30:59.047642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:59.048248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:59.048248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:59.049584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 722/1439] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:30:59.054792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239e1d-b92c-45cb-bc53-e2c21871cd28 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:59.055272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239e1d-b92c-45cb-bc53-e2c21871cd28 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:59.055392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239e1d-b92c-45cb-bc53-e2c21871cd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:59.097148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:59.122369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7a239e1d-b92c-45cb-bc53-e2c21871cd28 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:30:59.123591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7a239e1d-b92c-45cb-bc53-e2c21871cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:59.222088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 718/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:30:59.508519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:30:59.510677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:59.612590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c5bc1-79f4-4005-9794-4b6ca21789f2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:30:59.644911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:30:59.685810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:30:59.686412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:30:59.686689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:30:59.686958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:30:59.688529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:30:59.793987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:30:59.795415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d7343-ee1f-4d0f-a910-58f15c5bb794 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:30:59.814973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 723/1441] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:02.111188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd30bb-600d-49cf-8d6a-980bb75b4861 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:02.111729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd30bb-600d-49cf-8d6a-980bb75b4861 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:02.111729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd30bb-600d-49cf-8d6a-980bb75b4861 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:02.113007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 719/1442] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:02.118272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfa989-35b4-4818-87b8-2e0b3503ab21 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:02.118701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfa989-35b4-4818-87b8-2e0b3503ab21 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:02.118834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfa989-35b4-4818-87b8-2e0b3503ab21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:02.185602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-48dfa989-35b4-4818-87b8-2e0b3503ab21 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:02.186847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48dfa989-35b4-4818-87b8-2e0b3503ab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:02.199574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:02.397967 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 724/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:02.695750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:02.696037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:02.696318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:02.698223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 720/1444] 10.222.0.20 () {54 vars in 965 bytes} [Mon Sep 12 15:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:02.703951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9537-33ee-47c0-af17-f58823b33b9e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:02.704236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9537-33ee-47c0-af17-f58823b33b9e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:02.704510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c9537-33ee-47c0-af17-f58823b33b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:02.778980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca5c9537-33ee-47c0-af17-f58823b33b9e None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:02.780188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca5c9537-33ee-47c0-af17-f58823b33b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:02.853286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 725/1445] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Sep 12 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:02.862354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:02.863880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:02.966318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622802-13f8-4f9f-98e0-5ee4fdb28650 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:02.979045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:03.039538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:03.040139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:03.040421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:03.040696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:03.042347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:03.152943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:03.154427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9c27a-598f-41eb-bb39-90f1ab542fd0 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:03.172082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 721/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:06.074984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:06.075562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:06.075562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:06.076883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:06.082210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fda83-aefc-4ca8-a7a0-13e7569fb138 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:06.082614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fda83-aefc-4ca8-a7a0-13e7569fb138 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:06.082758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fda83-aefc-4ca8-a7a0-13e7569fb138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:06.129907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:06.148850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-296fda83-aefc-4ca8-a7a0-13e7569fb138 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:06.150133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-296fda83-aefc-4ca8-a7a0-13e7569fb138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:06.245729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:06.532871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:06.534457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:06.542993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:06.637418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f8d13-38b1-4ff0-ba47-770f4a60f151 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:06.712502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:06.713116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:06.713404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:06.713683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:06.715322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:06.774913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:06.825250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:06.826720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b191c95-4d29-4010-91b3-7682272b0c36 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:06.844652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 727/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:09.119140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:09.119718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:09.120090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:09.122483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:09.127712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cfa9e-f4fa-4d0e-9c87-cd96f0ed25ba None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:09.128163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cfa9e-f4fa-4d0e-9c87-cd96f0ed25ba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:09.128287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cfa9e-f4fa-4d0e-9c87-cd96f0ed25ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:09.192769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f16cfa9e-f4fa-4d0e-9c87-cd96f0ed25ba None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:09.193987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f16cfa9e-f4fa-4d0e-9c87-cd96f0ed25ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:09.289448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:09.576687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:09.578311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:09.690960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978fb00-359e-4b76-a5f3-500f80208a6e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:09.711565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:09.765228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:09.765863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:09.766151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:09.766429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:09.768030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:09.875566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:09.876999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed5ebb-c2da-4ca2-860d-bcaf86838a1e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:09.895519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 724/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:12.107224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:12.107769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:12.107769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:12.109100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:12.114578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4517-0f80-46c2-bea1-ab9cb426f368 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:12.115024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4517-0f80-46c2-bea1-ab9cb426f368 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:12.115174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4517-0f80-46c2-bea1-ab9cb426f368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:12.179033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:12.182374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b61c4517-0f80-46c2-bea1-ab9cb426f368 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:12.184123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b61c4517-0f80-46c2-bea1-ab9cb426f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:12.279572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:12.565845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:12.567367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:12.584595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:12.669094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d488299-aa09-44f8-81b2-d7248c7beded None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:12.742738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:12.743351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:12.744195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:12.744469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:12.746095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:12.814781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:12.854144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:12.855619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a2ad-acc2-4b8e-b312-98dfacd9faa4 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:12.873168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 730/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:15.068850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:15.069421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:15.069421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:15.070899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:15.076696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5659-6236-44c2-9239-b4a735ca4b59 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:15.077121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5659-6236-44c2-9239-b4a735ca4b59 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:15.077248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5659-6236-44c2-9239-b4a735ca4b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:15.145172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6ccd5659-6236-44c2-9239-b4a735ca4b59 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:15.146424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ccd5659-6236-44c2-9239-b4a735ca4b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:15.244247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:15.530935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:15.532497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:15.636263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8fc489-823a-4004-9f13-cdaa281b40cd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:15.664568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:15.710396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:15.711006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:15.711286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:15.711558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:15.713731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:15.820979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:15.823439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2f333-4324-4ad1-af3b-89577d5bb931 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:15.841050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 727/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:18.021918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:18.022618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:18.022618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:18.024045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:18.030062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144575c-a41a-4557-99d0-f962a0b6bb8c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:18.030524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144575c-a41a-4557-99d0-f962a0b6bb8c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:18.030922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144575c-a41a-4557-99d0-f962a0b6bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:18.097142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c144575c-a41a-4557-99d0-f962a0b6bb8c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:18.098393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c144575c-a41a-4557-99d0-f962a0b6bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:18.106629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:18.195520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:18.482230 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:18.483775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:18.511753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:18.585025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e72a06-e25f-48d1-a53b-4e4039036922 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:18.656592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:18.657176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:18.657456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:18.657744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:18.659316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:18.737239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:18.770057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:18.771494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868bde7-488a-482e-9589-6f4066fbf8bd service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:18.789326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 733/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:21.082323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:21.082902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:21.082902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:21.084211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:21.090043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730a18-c803-41c4-8dcc-fd3ca0f0c1e8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:21.090452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730a18-c803-41c4-8dcc-fd3ca0f0c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:21.090594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730a18-c803-41c4-8dcc-fd3ca0f0c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:21.155397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bf730a18-c803-41c4-8dcc-fd3ca0f0c1e8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:21.156629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf730a18-c803-41c4-8dcc-fd3ca0f0c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:21.250664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:31:21.538223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:21.539784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:21.643613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a459a1-0ce6-450a-8223-cf222c46bc02 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:21.684350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:21.840624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:21.841236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:21.841521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:21.841844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:21.843445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:21.952851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:21.954273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f36537-9c20-4aa5-b2c0-c894f7cc8097 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:21.972281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 730/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:24.156897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:24.157476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:24.157476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:24.158839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:24.164285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf38e15-9757-42ca-a33c-a689253bf7d0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:24.165298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf38e15-9757-42ca-a33c-a689253bf7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:24.165560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf38e15-9757-42ca-a33c-a689253bf7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:24.233356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-abf38e15-9757-42ca-a33c-a689253bf7d0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:24.234665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-abf38e15-9757-42ca-a33c-a689253bf7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:24.253632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:24.334974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:24.621787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:24.623324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:24.661615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:24.729277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f6d15-ecc8-423a-8b0e-85abf01161a3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:24.802917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:24.803638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:24.803915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:24.804181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:24.806400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:24.893319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:24.919842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:24.921303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a75a27-a2cf-4cd1-8ff3-878242896a4b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:24.939115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 736/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:27.162763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:27.163315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:27.163315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:27.165251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:27.170528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d73e-3b62-49d5-b98f-2aa3ea32a129 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:27.170974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d73e-3b62-49d5-b98f-2aa3ea32a129 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:27.171089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d73e-3b62-49d5-b98f-2aa3ea32a129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:27.237962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d841d73e-3b62-49d5-b98f-2aa3ea32a129 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:27.239200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d841d73e-3b62-49d5-b98f-2aa3ea32a129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:27.333389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:27.612672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:27.614246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:27.716629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed51ed-8746-430d-8b9e-58cc600526c1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:27.765323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:27.791203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:27.791803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:27.792080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:27.792350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:27.793950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:27.901408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:27.902788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9ef34-a433-40f4-a063-f597a979877a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:27.920353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 733/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:37.331897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:37.918222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=66 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:47.834201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa09fa-1d38-4418-aa22-3d917b028970 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:47.834739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa09fa-1d38-4418-aa22-3d917b028970 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:47.834739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa09fa-1d38-4418-aa22-3d917b028970 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:47.836103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 738/1471] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:47.841327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f775-93a4-4809-87ae-8873ff385593 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:47.841754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f775-93a4-4809-87ae-8873ff385593 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:47.841906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f775-93a4-4809-87ae-8873ff385593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:47.906815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5f21f775-93a4-4809-87ae-8873ff385593 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:47.908026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5f21f775-93a4-4809-87ae-8873ff385593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:48.005218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 734/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:48.304369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:48.304655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:48.304930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:48.306305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 739/1473] 10.222.0.20 () {54 vars in 966 bytes} [Mon Sep 12 15:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:48.311129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbed0-500a-4282-a89a-2a3d118b97d5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:48.311416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbed0-500a-4282-a89a-2a3d118b97d5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:48.311694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbed0-500a-4282-a89a-2a3d118b97d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:48.388041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b86dbed0-500a-4282-a89a-2a3d118b97d5 None None] MFA Rules not processed for user `9ae494b13b5743a2b56054d2fae338fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:48.389254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b86dbed0-500a-4282-a89a-2a3d118b97d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:48.410116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:48.466757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 735/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Mon Sep 12 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:48.474369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:48.475882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:48.581669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340e7e6-4fa7-4eb2-93b1-3579bde1efc2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:48.653344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:48.653970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:48.654253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:48.654526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:48.656116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:48.711214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:48.765651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:48.767114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a0bd-f4a2-459e-87ad-7724ee140c1e service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:48.786709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 740/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:51.026789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:51.027355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:51.027355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:51.028711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 736/1476] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:51.034088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615105e6-e0af-4120-b7bb-de41a1daffc8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:51.034488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615105e6-e0af-4120-b7bb-de41a1daffc8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:51.034633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615105e6-e0af-4120-b7bb-de41a1daffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:51.099563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-615105e6-e0af-4120-b7bb-de41a1daffc8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:51.100791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-615105e6-e0af-4120-b7bb-de41a1daffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:51.195939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 741/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:51.473228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:51.474787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:51.597916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee29df-98f4-4879-98a8-b75ca6122d4a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:51.648556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:51.695007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:51.695611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:51.695890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:51.696161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:51.696795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:51.698798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:31:51.698798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:31:51.698798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:31:51.699924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca303b1-82cb-4ba1-80d4-ef1cd63c785b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:51.718496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 737/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Sep 12 15:31:51 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:31:53.894828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:53.895393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:53.895393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:53.896713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 742/1479] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:53.901999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291343e2-b42d-4aa1-b8e7-ed708945618d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:53.902411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291343e2-b42d-4aa1-b8e7-ed708945618d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:53.902711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291343e2-b42d-4aa1-b8e7-ed708945618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:53.967739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-291343e2-b42d-4aa1-b8e7-ed708945618d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:53.968938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-291343e2-b42d-4aa1-b8e7-ed708945618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:54.065600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 738/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:54.361739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:54.363254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:54.372880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:54.506203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4d5ee-a1ff-472b-871f-2487b58d36ed None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:54.585527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:54.617903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:54.618502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:54.618783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:54.619056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:54.620643 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:54.710327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:54.712011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffbb62-bfbe-4fac-83e9-638df748f6b7 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:54.729527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 743/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:31:57.063047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:57.063648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:57.063648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:57.064969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 739/1482] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:31:57.070275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a7192-c227-46cf-8c30-7a0b9dc2d7c2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:57.070678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a7192-c227-46cf-8c30-7a0b9dc2d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:57.070840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a7192-c227-46cf-8c30-7a0b9dc2d7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:57.135651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-383a7192-c227-46cf-8c30-7a0b9dc2d7c2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:31:57.136859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-383a7192-c227-46cf-8c30-7a0b9dc2d7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:57.182885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:57.233201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 744/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:31:57.529012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:31:57.530590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:57.553912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:57.638723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955505f0-ea74-45c3-9ba1-dd842e632f1d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:31:57.713437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:31:57.714081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:31:57.714369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:31:57.714645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:31:57.716255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:31:57.789860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:31:57.826086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:31:57.827523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870dda0-de39-4bd7-bb5d-316b68b8f6c1 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:31:57.845255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 740/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:32:00.052421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:00.052989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:00.052989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:00.054876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 745/1485] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:32:00.060004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518601-17da-4e46-a013-006a3386bd40 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:00.060463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518601-17da-4e46-a013-006a3386bd40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:00.060569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5518601-17da-4e46-a013-006a3386bd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:00.125298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a5518601-17da-4e46-a013-006a3386bd40 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:32:00.127289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a5518601-17da-4e46-a013-006a3386bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:00.226452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 741/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:32:00.518789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:32:00.520325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:00.622252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b7b5d-fe1a-4b21-bea6-11607e207ea6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:32:00.695530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:00.697007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:32:00.697912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:00.698380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:00.698801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:00.700547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:00.805680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:32:00.807129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92a95c-97cb-4ca1-b106-2c75a5679ec6 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:32:00.824610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 746/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:32:03.057526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:03.058123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:03.058123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:03.059475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 742/1488] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:32:03.064663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d908-a763-4e6a-b48f-ad17bdc9cbd9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:03.065120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d908-a763-4e6a-b48f-ad17bdc9cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:03.065225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d908-a763-4e6a-b48f-ad17bdc9cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:03.132337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a759d908-a763-4e6a-b48f-ad17bdc9cbd9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:32:03.133550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a759d908-a763-4e6a-b48f-ad17bdc9cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:03.184155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:03.227590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 747/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:32:03.525891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:32:03.527464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:03.564167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:03.632395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e6f4-384b-4c38-81a9-3fa7d5501070 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:32:03.705571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:32:03.706198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:03.706479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:03.706750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:03.708328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:03.788306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:03.817639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:32:03.819167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bde0f-3ffd-4594-b2e6-5cf5f0ee3680 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:32:03.837392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 743/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:32:06.152671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:06.153248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:06.153248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:06.154645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 748/1491] 10.222.0.20 () {54 vars in 927 bytes} [Mon Sep 12 15:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:32:06.159968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8384-7c9e-4502-8d17-d7b564146d21 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:06.160419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8384-7c9e-4502-8d17-d7b564146d21 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:06.160524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a8384-7c9e-4502-8d17-d7b564146d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:06.227688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-359a8384-7c9e-4502-8d17-d7b564146d21 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:32:06.228888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-359a8384-7c9e-4502-8d17-d7b564146d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:06.326282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 744/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Sep 12 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:32:06.614024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:32:06.616149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:06.716333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e81d3-e84e-45f7-8159-16bb66768294 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:32:06.787574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:32:06.788173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:32:06.788448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:32:06.788715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:32:06.790312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:32:06.798770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:06.898113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:32:06.899475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614a07a-a610-407b-9f6c-6f3aaaf3135a service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:32:06.917127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 749/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:32:16.324689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=82 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:32:16.914883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=50 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:26.241219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:26.241875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:26.241875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:26.324517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:33:26.325787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:26.434858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 745/1494] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:33:26.443620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:26.444184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:26.444184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:26.523367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:33:26.524581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:26.624935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 750/1495] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:33:26.630359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:26.631629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:26.642980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc291e-9837-489a-96e0-878751deb731 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:26.707212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:26.742292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:26.742893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:26.743169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:26.743438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:26.744072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:33:26.744368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:26.745387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:26.745387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:33:26.745387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:33:26.746405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:26.757993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 746/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:33:26.769666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:26.778843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1c606-6ded-4f26-9260-b377e0230731 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:26.846761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:26.880202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:26.880804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:26.881085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:26.881354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:26.882008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:26.883740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:26.883740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:33:26.883740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:33:26.884933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:26.944886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 751/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:33:26 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:33:26.949143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:26.950735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:26.983647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05684d-1689-478e-837d-68fd7e3a405d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:27.077302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:27.077873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.078159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.078436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.079076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:27.080089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:27.080089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:33:27.080089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:33:27.081824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:27.123842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 747/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:33:26 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:33:27.127984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:27.129512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.191045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d02dd9-61b4-4fb3-b001-f712c3ee6201 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:27.274263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:27.285322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:27.285958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.286241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.286515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.287148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:27.288165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:27.288165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:33:27.288165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:33:27.289054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:27.301179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 752/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:33:27.305236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:27.306788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.318211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d2cbc-841f-4278-9f3f-3ccb3bb090ab admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:27.343792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:27.416892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:27.417497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.417814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.418095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.418738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:27.419757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:27.419757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:33:27.419757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:33:27.420646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb3efa-62a2-4e50-b7a4-452f07b49ad1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:27.432774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 748/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:33:27.437059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:27.438628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.472603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb88d0-f4c5-438b-9179-2e5b98895e6a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:27.564214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:27.565398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.565682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.565975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] PATH_INFO: `/v3/projects/688a1251810a4c5586771da4052a6653/users/d4dd208736cc40fa8342b2f2a7ff03ed/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.588371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688a1251810a4c5586771da4052a6653, user_id=d4dd208736cc40fa8342b2f2a7ff03ed, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:27.589419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:27.589419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:33:27.589419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:33:27.591394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0432b7-a717-4f25-ac07-a5b807623329 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:27.614855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 753/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:33:27 2022] PUT /identity/v3/projects/688a1251810a4c5586771da4052a6653/users/d4dd208736cc40fa8342b2f2a7ff03ed/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:33:27.627630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cc53c-b80c-4c06-8c1f-3d473196949f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.627922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cc53c-b80c-4c06-8c1f-3d473196949f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.628201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cc53c-b80c-4c06-8c1f-3d473196949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.685107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-391cc53c-b80c-4c06-8c1f-3d473196949f None None] MFA Rules not processed for user `d4dd208736cc40fa8342b2f2a7ff03ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:33:27.686338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-391cc53c-b80c-4c06-8c1f-3d473196949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.765861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 749/1502] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:33:27.779217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.779504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.779805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.781143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 754/1503] 10.222.0.20 () {54 vars in 964 bytes} [Mon Sep 12 15:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:33:27.786319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d82f-a452-484b-adbb-efc393fc1bf5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:27.786606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d82f-a452-484b-adbb-efc393fc1bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:27.786880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d82f-a452-484b-adbb-efc393fc1bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:27.817943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:27.862384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45a7d82f-a452-484b-adbb-efc393fc1bf5 None None] MFA Rules not processed for user `44a33c23a10f4ae5a008e1d4d4973355`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:33:27.863580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45a7d82f-a452-484b-adbb-efc393fc1bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.937175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 750/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Sep 12 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:33:27.947064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:27.948561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:27.998976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:28.052396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fac6db-39b5-4a4c-8ca9-0e0d39f51b1c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:28.124941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:28.125539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:28.125860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:28.126177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:28.127752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:28.224536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:28.225934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:28.229344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:28.244187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 755/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:33:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:33:28.289189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa533-efa3-46de-bf71-332128ca1d41 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:28.289481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa533-efa3-46de-bf71-332128ca1d41 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:28.289792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faa533-efa3-46de-bf71-332128ca1d41 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:28.291050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 751/1506] 10.222.0.20 () {52 vars in 830 bytes} [Mon Sep 12 15:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:33:28.294665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:28.295880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:28.395286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ce07a-c17a-4734-9825-40f175677f8e service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:28.486600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:28.487198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:28.487457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:28.487728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] PATH_INFO: `/v3/users/d4dd208736cc40fa8342b2f2a7ff03ed` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:28.500337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4dd208736cc40fa8342b2f2a7ff03ed)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:28.501355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:28.501355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:33:28.501355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:33:28.503112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:28.512729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:28.569813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4dd208736cc40fa8342b2f2a7ff03ed'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:28.570250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4dd208736cc40fa8342b2f2a7ff03ed'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:28.571030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4dd208736cc40fa8342b2f2a7ff03ed'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:28.584360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4dd208736cc40fa8342b2f2a7ff03ed'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:28.606225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4dd208736cc40fa8342b2f2a7ff03ed'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:28.621334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 756/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:33:28 2022] DELETE /identity/v3/users/d4dd208736cc40fa8342b2f2a7ff03ed => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:33:28.625909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:28.626383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:28.626495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:28.683375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:33:28.684578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:28.764368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:28.777123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 752/1508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:33:28.786888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:28.788142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:28.887053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c17e7-585a-42b0-bf16-bda994601f40 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:28.942510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:28.963700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:28.964324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:28.964603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:28.964874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:28.966480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:29.073014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:29.074470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cb347-3753-4657-ba3e-017004daa243 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:29.092573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 757/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:33:29.437664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:33:29.439009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:33:29.451104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b3c3-0301-43c0-bec7-a3aabb74ea63 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:33:29.550360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:33:29.551031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:33:29.551313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:33:29.551587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] PATH_INFO: `/v3/projects/688a1251810a4c5586771da4052a6653` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:33:29.562254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688a1251810a4c5586771da4052a6653)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:33:29.563286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:33:29.563286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:33:29.563286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:33:29.564488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:33:29.630104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688a1251810a4c5586771da4052a6653', 'request_id': 'req-b369346e-c2e3-40ad-bbcd-3408eab5f720'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:29.643041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688a1251810a4c5586771da4052a6653', 'request_id': 'req-b369346e-c2e3-40ad-bbcd-3408eab5f720'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:29.649299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688a1251810a4c5586771da4052a6653', 'request_id': 'req-b369346e-c2e3-40ad-bbcd-3408eab5f720'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:29.677128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] The token cache is being invalidate because project 688a1251810a4c5586771da4052a6653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:33:29.677427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:29.677895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b369346e-c2e3-40ad-bbcd-3408eab5f720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:33:29.679300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 753/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:33:29 2022] DELETE /identity/v3/projects/688a1251810a4c5586771da4052a6653 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:33:39.090358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=62 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:33:39.674549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:35:01.013980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:01.014594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:01.014594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:01.015927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:01.020764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a38d-6bc2-44d1-9b14-96f0221cd3ed None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:01.021175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a38d-6bc2-44d1-9b14-96f0221cd3ed None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:01.021356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a38d-6bc2-44d1-9b14-96f0221cd3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:01.102383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6815a38d-6bc2-44d1-9b14-96f0221cd3ed None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:01.103610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6815a38d-6bc2-44d1-9b14-96f0221cd3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:01.212972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:35:01.366219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:01.368327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:01.484668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dde82-fcf9-4fd1-b7b4-97732e25b6d3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:01.490181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:01.490785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:01.491072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:01.491352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:01.499343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:01.500921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a490-f936-4068-8ade-46421afe98b1 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:01.510261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 759/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:02.602903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540055d2-e08c-41fb-a973-8e9769bbee4a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:02.603464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540055d2-e08c-41fb-a973-8e9769bbee4a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:02.603464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540055d2-e08c-41fb-a973-8e9769bbee4a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:02.604817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:02.611646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07b7fc-3182-4537-a42e-0680b0a369e1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:02.612078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07b7fc-3182-4537-a42e-0680b0a369e1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:02.612203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07b7fc-3182-4537-a42e-0680b0a369e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:02.631605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3c07b7fc-3182-4537-a42e-0680b0a369e1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:02.632776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3c07b7fc-3182-4537-a42e-0680b0a369e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:02.640668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:02.792481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540055d2-e08c-41fb-a973-8e9769bbee4a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:02.806589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540055d2-e08c-41fb-a973-8e9769bbee4a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:02.811966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:02.812572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:02.812853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:02.813130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:02.821265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:02.822878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3320a5-614a-46b9-9201-8bc650bb733f service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:02.832483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 756/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:03.098346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec89a8-96f4-43ff-8b95-a6eccc9b5020 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:03.098639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec89a8-96f4-43ff-8b95-a6eccc9b5020 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:03.098914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec89a8-96f4-43ff-8b95-a6eccc9b5020 None None] PATH_INFO: `/v3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:03.100204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 761/1517] 10.222.0.15 () {54 vars in 925 bytes} [Mon Sep 12 15:35:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:35:03.114398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cb49-1f66-44d3-84e5-911202bcc8ca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:03.114688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cb49-1f66-44d3-84e5-911202bcc8ca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:03.114968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1cb49-1f66-44d3-84e5-911202bcc8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:03.173877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7be1cb49-1f66-44d3-84e5-911202bcc8ca None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:03.175094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7be1cb49-1f66-44d3-84e5-911202bcc8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:03.192885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:35:03.249030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 757/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 12 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:06.842863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed160f3-b575-4108-b1d6-99a60962202c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:06.843827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed160f3-b575-4108-b1d6-99a60962202c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:06.843941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed160f3-b575-4108-b1d6-99a60962202c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:06.845245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 762/1519] 10.222.0.20 () {54 vars in 961 bytes} [Mon Sep 12 15:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:06.850282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f08bd-74ce-4128-bed7-e23ad56a23dd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:06.850741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f08bd-74ce-4128-bed7-e23ad56a23dd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:06.850895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f08bd-74ce-4128-bed7-e23ad56a23dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:06.870911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-088f08bd-74ce-4128-bed7-e23ad56a23dd None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:06.872110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-088f08bd-74ce-4128-bed7-e23ad56a23dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:06.880641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 758/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Sep 12 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:06.888109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed160f3-b575-4108-b1d6-99a60962202c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:06.901785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed160f3-b575-4108-b1d6-99a60962202c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:06.907337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:06.907920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:06.908204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:06.908479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:06.916581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:06.918175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef64db-65b7-4001-817c-913321e73a31 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:06.927607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 763/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:14.291992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4101b8-a826-4e47-8fe6-f5a3660bc939 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:14.292588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4101b8-a826-4e47-8fe6-f5a3660bc939 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:14.292588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4101b8-a826-4e47-8fe6-f5a3660bc939 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:14.294173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:14.300381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a581eb4-71e3-4e4c-9dc7-faa7d77f0a82 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:14.300831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a581eb4-71e3-4e4c-9dc7-faa7d77f0a82 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:14.300960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a581eb4-71e3-4e4c-9dc7-faa7d77f0a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:14.321051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3a581eb4-71e3-4e4c-9dc7-faa7d77f0a82 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:14.322299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a581eb4-71e3-4e4c-9dc7-faa7d77f0a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:14.330745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:14.493604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4101b8-a826-4e47-8fe6-f5a3660bc939 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:14.508768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4101b8-a826-4e47-8fe6-f5a3660bc939 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:14.514157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89987341-737d-439e-94be-db10708b9f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:14.514755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987341-737d-439e-94be-db10708b9f65 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:14.515037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987341-737d-439e-94be-db10708b9f65 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:14.515313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987341-737d-439e-94be-db10708b9f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:14.523187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89987341-737d-439e-94be-db10708b9f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:14.524777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89987341-737d-439e-94be-db10708b9f65 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:14.534142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 760/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:15.613899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d114d8-702c-4867-af56-c0727fe4b48b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:15.614461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d114d8-702c-4867-af56-c0727fe4b48b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:15.614461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d114d8-702c-4867-af56-c0727fe4b48b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:15.615806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 15:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:15.622132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb160a-ae4b-46c4-8136-e658471d5019 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:15.622553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb160a-ae4b-46c4-8136-e658471d5019 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:15.622644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb160a-ae4b-46c4-8136-e658471d5019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:15.642913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-21cb160a-ae4b-46c4-8136-e658471d5019 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:15.644115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-21cb160a-ae4b-46c4-8136-e658471d5019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:15.652009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:15.803644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d114d8-702c-4867-af56-c0727fe4b48b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:15.818352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d114d8-702c-4867-af56-c0727fe4b48b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:15.823900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:15.824480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:15.824758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:15.825030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:15.833464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:15.835068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fca42-29de-487e-aa13-a1bb5f66863b service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:15.844090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 766/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:19.944379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055d23-63d8-4120-9394-12587addbeed None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:19.944937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055d23-63d8-4120-9394-12587addbeed None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:19.944937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a055d23-63d8-4120-9394-12587addbeed None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:19.946579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:19.951737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1eb8-d3dd-4e14-ba6e-d524b9429ea0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:19.952142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1eb8-d3dd-4e14-ba6e-d524b9429ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:19.952287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1eb8-d3dd-4e14-ba6e-d524b9429ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:19.972546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-af0d1eb8-d3dd-4e14-ba6e-d524b9429ea0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:19.973669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af0d1eb8-d3dd-4e14-ba6e-d524b9429ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:19.981337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:20.265466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a055d23-63d8-4120-9394-12587addbeed None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:20.267821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1a055d23-63d8-4120-9394-12587addbeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:20.345468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a055d23-63d8-4120-9394-12587addbeed None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:20.349014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:20.349618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:20.349940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:20.350202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:20.358965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:20.360926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93c79e-6b0a-45b2-93cc-12cdbb8524f9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:20.370470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 763/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:22.531836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9cb12-c642-4325-b0c3-632543250595 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:22.532459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9cb12-c642-4325-b0c3-632543250595 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:22.532459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9cb12-c642-4325-b0c3-632543250595 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:22.533952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:22.539020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe33fc-39df-408a-9bb2-55b580832528 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:22.539468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe33fc-39df-408a-9bb2-55b580832528 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:22.539570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe33fc-39df-408a-9bb2-55b580832528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:22.559491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4fe33fc-39df-408a-9bb2-55b580832528 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:22.560692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4fe33fc-39df-408a-9bb2-55b580832528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:22.568535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:22.860310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9cb12-c642-4325-b0c3-632543250595 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:22.870848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9cb12-c642-4325-b0c3-632543250595 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:22.874272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:22.874856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:22.875197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:22.875471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:22.884102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:22.885523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cb26f-e003-4410-9d17-1e39a6e6c61e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:22.895201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 769/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:26.478256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:26.478867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:26.478867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:26.480184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:26.486046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3052d-bfb3-4596-bfac-14b2ca17120f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:26.486456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3052d-bfb3-4596-bfac-14b2ca17120f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:26.486609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3052d-bfb3-4596-bfac-14b2ca17120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:26.506597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-19c3052d-bfb3-4596-bfac-14b2ca17120f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:26.507804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-19c3052d-bfb3-4596-bfac-14b2ca17120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:26.515640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:26.818205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:26.819934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:26.884186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:35:26.898033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d71e4e-47e4-406d-a164-43d687e16fe0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:26.901544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:26.902185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:26.902469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:26.902745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:26.912052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:26.913480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598fd03-b44e-4b26-b5c5-82ea556a4f2c service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:26.923357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 766/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:29.079799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:29.080362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:29.080362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:29.081722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 15:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:35:29.086999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb941b-8f1d-47da-924b-fba21f671534 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:29.087413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb941b-8f1d-47da-924b-fba21f671534 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:29.087554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb941b-8f1d-47da-924b-fba21f671534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:29.108478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0afb941b-8f1d-47da-924b-fba21f671534 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:35:29.109766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0afb941b-8f1d-47da-924b-fba21f671534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:29.119228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 15:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:35:29.422197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:35:29.423869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:35:29.431302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:35:29.438162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ee88-5c17-4381-bf93-cb72c09c8b6d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:35:29.441730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:35:29.442357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:35:29.442641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:35:29.442917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:35:29.452176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:35:29.453631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c334d4-dd9b-4c62-9840-a4a4e0c53ce9 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:35:29.462923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 772/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:35:39.103501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=11 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:35:39.458734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:54.939759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21d23c-6e0f-4722-b5f7-f8f29b1b82df None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:54.940488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21d23c-6e0f-4722-b5f7-f8f29b1b82df None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:54.940488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21d23c-6e0f-4722-b5f7-f8f29b1b82df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:54.946520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ef9c-a944-4a7c-9b7a-815a1d4b4c3f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:54.946991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ef9c-a944-4a7c-9b7a-815a1d4b4c3f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:54.947092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ef9c-a944-4a7c-9b7a-815a1d4b4c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:54.974561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-aaf1ef9c-a944-4a7c-9b7a-815a1d4b4c3f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:54.974909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cb21d23c-6e0f-4722-b5f7-f8f29b1b82df None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:54.975771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aaf1ef9c-a944-4a7c-9b7a-815a1d4b4c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:54.976032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb21d23c-6e0f-4722-b5f7-f8f29b1b82df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:54.990815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 768/1540] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:54.998518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 773/1541] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:36:55.004729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.005029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.005315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.008079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.008373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.008673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.035850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.037100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.043367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.044691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.055812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 769/1542] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.062262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.066091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 774/1543] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:36:55.070872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.077951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b39f7-415e-4969-b94c-ec268ed48d71 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.086628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e659f1-88d1-4c4b-a881-bbc8a1029ace None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.094395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.095238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.095534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.095935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.096837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.097155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.098928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76c05e-a645-404d-b0ce-29c1528ff4d7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.102376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.102997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.103282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.103557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.104210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.104510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.105528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:36:55.105528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:36:55.105528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:36:55.106505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.114377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 770/1544] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:36:55.117587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.117943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.118188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.136382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 775/1545] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:36:55.137270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.154102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8ffe-111e-4d99-b756-394738c8a64a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.162128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.163518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.164669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.165607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.166140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.166628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.167502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.169690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90ee0-3758-49eb-8f00-3c380721fd33 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.189399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 771/1546] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.190151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.203723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339562b-72c9-48ce-b8d8-5ca6322be226 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.223131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.223886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.224178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.224459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.225114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.226191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:36:55.226191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:36:55.226191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:36:55.227542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d03c9-ea1b-4513-9aaf-d32619d76363 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.271932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 776/1547] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/projects => generated 393 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.272965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.273424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.275285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.286707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 772/1548] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.288223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.288522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.288805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.324234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.325467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.331746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.333420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.467136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 773/1549] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.469785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.477941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 777/1550] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.479294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.481668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3ae67-4c19-4760-af1c-c4c812480d27 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.492401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.493023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.493305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.493575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.493918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793072e-0653-49a5-b1ff-02ea5cac7f9a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.494302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.496678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3f636-a6bd-4a45-beb4-2c90f3ed2ddd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.499572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.500186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.500474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.501354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.502283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.503305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:36:55.503305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:36:55.503305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:36:55.504514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ca7ab-fd56-49b2-9436-58296b04c335 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.521834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 774/1551] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/users => generated 297 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.523228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4d0f-4ce2-4dd1-b9d6-d593506cabe7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.523517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4d0f-4ce2-4dd1-b9d6-d593506cabe7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.523792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4d0f-4ce2-4dd1-b9d6-d593506cabe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.532059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 778/1552] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/users => generated 293 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.533287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.533582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.533891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.547199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d37a4d0f-4ce2-4dd1-b9d6-d593506cabe7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.548408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d37a4d0f-4ce2-4dd1-b9d6-d593506cabe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.553723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.554959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.556203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 775/1553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.557369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184169-a20e-4448-a746-281584bb69dc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.557649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184169-a20e-4448-a746-281584bb69dc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.557948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e184169-a20e-4448-a746-281584bb69dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.563251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 779/1554] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.563975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.573761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c3ede-12ac-49d3-9ed4-6f2de7ffbd15 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.578652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6e184169-a20e-4448-a746-281584bb69dc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.579404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.579842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6e184169-a20e-4448-a746-281584bb69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.580052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.580456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.580746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.581387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.581698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.583336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd734907-8578-464d-af2c-66f3faf8fbd3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.587942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 776/1555] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.589106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.589386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.589655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.597550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 780/1556] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:55.598670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.598965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.599246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.616161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.617368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.618528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.619684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.625172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 777/1557] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.625949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.628054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 781/1558] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.628772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.638444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f9f1-15b6-41b3-8bf3-1ca48b178118 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.643609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc1e79-743c-483c-a168-c82b6267dbf4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.644127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.644860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.645145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.645422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.646158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.647779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00a842-3fd9-4e74-a08e-a2abd1d54726 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.648221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.648823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.649287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.649571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.650626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.652221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f94668-b86f-414d-892e-8bf4a0c7c43d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.658449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 782/1559] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:55.659602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.659885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.660154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.665042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 778/1560] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:55.666295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.666585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.666864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.680081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.681434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.686931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.688113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.691362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 783/1561] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.692061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.697222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 779/1562] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.697910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.701984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799d38-2b58-4390-adbb-66be495ee7ee None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.707210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.707796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.708076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.708182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19186cdf-095e-46d9-8eb4-6396666a4523 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.708346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.708981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.709280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.710905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21cc38-793b-45fb-9946-3de4b25b5a89 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.712654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.713553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.713553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.713846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.714453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.716326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e7e9a-5602-4ddf-a7cc-519b8efd580e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.722988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 784/1563] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:55.724145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.724434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.724716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.744408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.745835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.751057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 780/1564] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.751057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.751057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.751057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.769070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:55.770289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:55.773942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:55.796613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:55.813810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 785/1565] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.815119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.825452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468d31b-d69e-4a03-902b-f6836e8a7fcb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.830778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.831375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.831666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.831946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.832590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.833871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 781/1566] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:55.834318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.834861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.844782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5f5c-5c3d-4b74-a3f7-3d04d65de73b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.848155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 786/1567] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:55.848155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.852113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.852718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.852998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.853272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.853956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.855541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.855972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195f52-cdac-495c-82f7-a917b347c8e4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.860573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.861154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.861439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.861738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.862383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.862693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.864361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.868949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 782/1568] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:55.868949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.875644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1070d-dee2-454f-b286-43aabf1787ca admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.880550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.881136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.881548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 787/1569] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:55.881677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.881677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.882211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.882323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:55.882749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.884330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.892234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63923c31-5962-4563-ad92-cf93271af030 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.896833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.897423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.897723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.898019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.898646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.899867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 783/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:55.900552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.900661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.916158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4907-49b5-4937-b213-1868b6894c6d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.920889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:55.921499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:55.921870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:55.922158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:55.922783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:55.924724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:55.932270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 788/1571] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.933097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.949675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 784/1572] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:55 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:55.950514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:55.960247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74901d49-14a4-463e-8f7e-8f0e02e08f71 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:55.995792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c231b6-eb3e-4b4a-83d9-079915b99151 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.000899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.001499 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.001815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.002087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.002730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:56.003035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.004623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.016088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.016699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.016895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 789/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:56.016999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.017243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.017757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.017923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:36:56.018231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.019890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.039135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 785/1574] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:36:56.039135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.039439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4843-18f4-4025-a549-bc40578285ef admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.039439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.039439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.039439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.039808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] PATH_INFO: `/v3/projects/6661c7cb48d947439a324c3f795fd57c/users/1ca7bef620284345a977088f9a0d2a07/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.045237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977bfe-0f79-4b90-9558-b7b361fea348 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.050281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.050936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.051242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.051509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] PATH_INFO: `/v3/projects/555943f87fe544d7a45eb6979069230d/users/d718e35636464834bc20a6eda0599cfc/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.056530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6661c7cb48d947439a324c3f795fd57c, user_id=1ca7bef620284345a977088f9a0d2a07, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.059268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.067569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555943f87fe544d7a45eb6979069230d, user_id=d718e35636464834bc20a6eda0599cfc, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.068787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:36:56.068787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:36:56.068787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:36:56.070865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.073540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 790/1575] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:56 2022] PUT /identity/v3/projects/6661c7cb48d947439a324c3f795fd57c/users/1ca7bef620284345a977088f9a0d2a07/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:56.074403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.087239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 786/1576] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:56 2022] PUT /identity/v3/projects/555943f87fe544d7a45eb6979069230d/users/d718e35636464834bc20a6eda0599cfc/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:56.088030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.131137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:56.143698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896faca-ad07-467e-83ea-6359e5a2fa2c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.148224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.148749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:56.148896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.149093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.149368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.150046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.152100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.153091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61045755-3779-4982-a62f-8818cd9f0005 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.159674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.160286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.160568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.160837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.161935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.164248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.181533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 791/1577] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.181533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.192885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 787/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.193953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.244427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edff27-093e-477b-aec2-43fa3f44c08b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.256187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167db07-43db-4c40-9eec-1b97e89a7b4b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.260689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.261293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.261573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.261878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.262512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.264115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.274613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 788/1579] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.274613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.284935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad2e4-0523-4a18-9999-e11a34340892 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.289449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.290062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.290344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.290617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.291253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.293117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.304141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.304782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.305132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.305413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.306089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.308005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c8ae8-bcbb-42ef-91bb-15f59cd030c0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.321754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 789/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.322268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.336603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 792/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.336603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69d19-efdc-4177-85d7-1f99e1f2aa7b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.336603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69d19-efdc-4177-85d7-1f99e1f2aa7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.336603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69d19-efdc-4177-85d7-1f99e1f2aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.353912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-52f69d19-efdc-4177-85d7-1f99e1f2aa7b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:56.355111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-52f69d19-efdc-4177-85d7-1f99e1f2aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:56.392305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd21fb-159b-4c0c-94d3-d76adc35df18 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.397047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.397653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.397993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.398444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.399081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.401017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.412522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 793/1582] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:56.413575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.413958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.414231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.434362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 790/1583] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.434624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:56.435220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.435857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:56.446329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:56.481427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:56.498109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 794/1584] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:56.502117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.502222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a579-9a4e-459c-bc1d-30b04d4aea3d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.510812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.510983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7043-f6e3-4189-bc9e-4bd28968e832 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.511746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.512031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.512305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.512941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.515012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.515718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.516314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.516595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.516868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.517484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.519418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.536898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 791/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.537979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.544126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 795/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/users => generated 294 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.544126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.549628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ac188-0085-453b-9f8c-df7c8ff10f7b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.553833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdce4a-680e-47e9-8f95-b04af9775985 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.554333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.555024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.555346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.555628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.556268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.557910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.558590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.559207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.559491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.559769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.560412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.563065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.569408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 792/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.570071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.579893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98439d-7fea-4e7e-a62e-7a5ce68a8472 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.584701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.585290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.585575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.585884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.586669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 796/1588] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.586792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.587294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.588089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.598421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 793/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.599196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.609173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b7c5b-9c99-443a-bb26-84f1bc314dc4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.613997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.614966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.615255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.615532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.616541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.618464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.642946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 794/1590] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:56.642946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.644907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c595bf-9f5c-4f11-a5c5-460f8e1a2d2c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.648706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.649355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.649642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.650033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.651281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafb776-605f-4130-8935-e6c17996e3ee admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.655095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.655876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.656162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.656441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.659221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.660664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.665347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.666810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.670238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 797/1591] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:56.671578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.680821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e25487-e20c-4b40-bc84-4fd1afaf3101 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.684182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 795/1592] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:56.685176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.686126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.687290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.687572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.687842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.688474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.690094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.696936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1689b-ddb9-448a-9879-b9d515373a34 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.704287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 798/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.704287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.705676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.706185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.706473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.706753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.707692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.709993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.713046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e79b8-d550-42b2-ac27-ae7387abb391 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.719409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.720063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.720347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.720621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] PATH_INFO: `/v3/projects/9a2f7a7e414840b6a8745b8c6f355e4f/users/a14a86493cb3425b977f1f660b05a905/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.722617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 796/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.723507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.735620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401312c1-0831-447a-bdcd-c6b89127f3da admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.740676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.741324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.741613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.741928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.742595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2f7a7e414840b6a8745b8c6f355e4f, user_id=a14a86493cb3425b977f1f660b05a905, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.742719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.744396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.745239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.760473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 797/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.760473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.761838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 799/1596] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:56 2022] PUT /identity/v3/projects/9a2f7a7e414840b6a8745b8c6f355e4f/users/a14a86493cb3425b977f1f660b05a905/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:56.762996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.772261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceab0f-647b-46b5-aaa6-5061d9271e0e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.825807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:56.843485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b9a79-9005-41cb-9fc7-7874d3d0b6c8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.849009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.849719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.849858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.850020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.850301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.850479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.850891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.850989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.851318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.851962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.852698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.853602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.869774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 800/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.869774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.870092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 798/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.870092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.879642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19343-3a64-4217-b311-40439d91161a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.883319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a901fa8-36e3-4703-a3ef-b3cc7367bc5d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.885310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.886043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.886326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.886986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.887719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.888586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.889184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.889298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.889471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.890401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.891123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.892882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b5ec0-aa8b-4afe-ade5-4281cd369df2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.906265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 799/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.906913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 801/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.907133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.907219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.907219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.907219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.922511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47683-21e9-40ad-9600-afce2f169c9a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.928278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.931036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.931390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.931681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.932473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.934312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:56.940716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] MFA Rules not processed for user `a14a86493cb3425b977f1f660b05a905`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:56.942088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:56.949818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 800/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:56.949818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:56.962037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58819150-968b-4bc2-aedc-ae3b597851dc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:56.968267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:56.968926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:56.969215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:56.969495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] PATH_INFO: `/v3/projects/d1f5593b589846538753b8919b8da579/users/a1a9b41251e146eba110fa1e2b1fc9c5/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:56.988461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f5593b589846538753b8919b8da579, user_id=a1a9b41251e146eba110fa1e2b1fc9c5, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:56.991699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.007127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 801/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:56 2022] PUT /identity/v3/projects/d1f5593b589846538753b8919b8da579/users/a1a9b41251e146eba110fa1e2b1fc9c5/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:57.007844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.017407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:57.027667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 802/1603] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.029283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.043425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc786442-05c5-4a51-ae13-e13d2d1eb2dc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.096914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b8ee5-1aba-49eb-972c-852b150c8957 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.101943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.102567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.102864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.103139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.103797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.105652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.121084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 802/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:57.135110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.147252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.147966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.148288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.151604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3686eb-3154-4745-9754-21e0853e54a8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.153812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] PATH_INFO: `/v3/projects/802e0a1af12748448cbcfecaced16aa6/users/930cbe41fc0741cfa5fbd81645181418/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.161425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.162848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.163187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.163758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] PATH_INFO: `/v3/projects/fb0fd295845f4e7a9d5f3eb5f1073085/users/bd428de0e4a54621bd6d55309ada03af/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.183306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802e0a1af12748448cbcfecaced16aa6, user_id=930cbe41fc0741cfa5fbd81645181418, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.190001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340f52-06a0-48f8-a37c-7ba09892c782 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.190130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0fd295845f4e7a9d5f3eb5f1073085, user_id=bd428de0e4a54621bd6d55309ada03af, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.193469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.215011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 803/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:57 2022] PUT /identity/v3/projects/fb0fd295845f4e7a9d5f3eb5f1073085/users/bd428de0e4a54621bd6d55309ada03af/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:57.216401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.236669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 803/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:57 2022] PUT /identity/v3/projects/802e0a1af12748448cbcfecaced16aa6/users/930cbe41fc0741cfa5fbd81645181418/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:57.237388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09e11-6a8b-4175-9427-2a0ac610b1d3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.239431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09e11-6a8b-4175-9427-2a0ac610b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.239723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e09e11-6a8b-4175-9427-2a0ac610b1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.266754 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-76e09e11-6a8b-4175-9427-2a0ac610b1d3 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.268445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-76e09e11-6a8b-4175-9427-2a0ac610b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.321724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:57.335796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63222066-3ce9-4241-80d2-ee939d1e46dd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.359506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 804/1607] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.359506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc02-4451-4975-86c7-dcb0b0675fff None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.364193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc02-4451-4975-86c7-dcb0b0675fff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.364548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dcc02-4451-4975-86c7-dcb0b0675fff None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.368413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 805/1608] 10.222.0.20 () {54 vars in 962 bytes} [Mon Sep 12 15:36:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:36:57.369621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcc02-4451-4975-86c7-dcb0b0675fff None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.385088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dcc02-4451-4975-86c7-dcb0b0675fff None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.391341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.392789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.393080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.393422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] PATH_INFO: `/v3/projects/71cff1e605f14117800fa9ee36eb4f1e/users/66e3d6e996a54f5a9bc08a3dbfefd1a7/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.413511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cff1e605f14117800fa9ee36eb4f1e, user_id=66e3d6e996a54f5a9bc08a3dbfefd1a7, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.417392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b35a09-f97e-4027-aab4-eb3b7b0d5e44 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.433472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.434286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.434624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.435001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] PATH_INFO: `/v3/projects/156e628f69d7448dbd0cb45c6dd3ac54/users/5d1ab08e56ca4ad692869347a8cd7e6b/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.436384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 806/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:57 2022] PUT /identity/v3/projects/71cff1e605f14117800fa9ee36eb4f1e/users/66e3d6e996a54f5a9bc08a3dbfefd1a7/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:57.439054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.439712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.440000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.468916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156e628f69d7448dbd0cb45c6dd3ac54, user_id=5d1ab08e56ca4ad692869347a8cd7e6b, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.471391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] MFA Rules not processed for user `bd428de0e4a54621bd6d55309ada03af`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.472256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effac137-f78d-4bbc-9bb7-b787811f5921 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.472624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.488272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 804/1610] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:57 2022] PUT /identity/v3/projects/156e628f69d7448dbd0cb45c6dd3ac54/users/5d1ab08e56ca4ad692869347a8cd7e6b/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:57.489262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d850-8115-4b82-a217-2c85b145be44 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.489552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d850-8115-4b82-a217-2c85b145be44 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.489859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d850-8115-4b82-a217-2c85b145be44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.511051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cdd8d850-8115-4b82-a217-2c85b145be44 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.512333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cdd8d850-8115-4b82-a217-2c85b145be44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.516928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:57.555410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 807/1611] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.556720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.580571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 805/1612] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.581923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bcdb2-c027-4f69-a7ad-7beca9ca95b3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.582209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bcdb2-c027-4f69-a7ad-7beca9ca95b3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.582488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bcdb2-c027-4f69-a7ad-7beca9ca95b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.633099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-be4bcdb2-c027-4f69-a7ad-7beca9ca95b3 None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.634517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-be4bcdb2-c027-4f69-a7ad-7beca9ca95b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.634657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf12a7-a3e8-4f3e-a1ba-cbe85fd331f0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.692576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.693389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.693601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.693892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.703792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.705248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353319c1-47f2-44ab-88aa-dfa99ba61d2a service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.711368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 806/1613] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.712694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.712987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.713263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.716694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 808/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:57.718653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.718949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.719228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.739869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.741146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.746384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] MFA Rules not processed for user `66e3d6e996a54f5a9bc08a3dbfefd1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.747796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.751750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 809/1615] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.752915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.760723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:57.767001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e199e-e1f2-4035-8baf-5fa2744553e9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.782872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.783607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.784141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.784576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.797386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.799853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c390e-f6bd-4d1e-b120-97e481b9b7b5 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.813502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 810/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:57.815042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1599f-9b4c-405b-8cac-09a4e0d9e6cb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.815421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1599f-9b4c-405b-8cac-09a4e0d9e6cb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.815800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1599f-9b4c-405b-8cac-09a4e0d9e6cb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.825944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 807/1617] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.825944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.826249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 811/1618] 10.222.0.20 () {54 vars in 962 bytes} [Mon Sep 12 15:36:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:36:57.826249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1599f-9b4c-405b-8cac-09a4e0d9e6cb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.846068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1599f-9b4c-405b-8cac-09a4e0d9e6cb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.847277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174e5c2-e250-4541-9f6b-2bc9c0c8fe5a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.854460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.855232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.855802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.856302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.861264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.863504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.864059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.864518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.877358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:57.881430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.883851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bf3b4-98a6-4dae-b9dd-410447fba550 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.906779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 812/1619] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:57.910757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.913304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.913610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.946113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:57.947951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:57.961033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.962004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 813/1620] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:57.962972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:57.974821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db39fbb-3261-415e-8c28-527e143371dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:57.980539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:57.981409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:57.981793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:57.982100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:57.982707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:57.986707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:57.998788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.034667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 814/1621] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:36:57 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:58.036207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.038426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 808/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:58.039463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.070008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7387a9-3ef7-478a-83ab-c10dfc32040e service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.155953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4244ab-e702-4273-8d86-2d80bef031c6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.168271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.169367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.169732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.170017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.223061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.224293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.224990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.226244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.227121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.230067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.274790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 809/1623] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:36:58 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:36:58.278218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.286362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.289210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.312254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 815/1624] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:58.314722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64c589-a3f6-47be-9b89-10628a461dc1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.324468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.325022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.325313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.325646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.327371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.329068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4f405-2fc3-44cc-a284-594415ab6c99 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.361378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 810/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:58.372174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.388787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9155-0f03-431e-b77f-bdb69f4b323c service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.406941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.407640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.410767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.410971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] PATH_INFO: `/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2/users/ae0afd5cae1b4bb5a4a6cc058aabadbb/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.426091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34465bf6-5a9b-419c-a2e1-2e1f65044ac1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.426448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34465bf6-5a9b-419c-a2e1-2e1f65044ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.426659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34465bf6-5a9b-419c-a2e1-2e1f65044ac1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.428542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 811/1626] 10.222.0.20 () {54 vars in 912 bytes} [Mon Sep 12 15:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:36:58.434104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.434389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.434669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.446376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc75273e8bd74a7a896ff0e9feb781e2, user_id=ae0afd5cae1b4bb5a4a6cc058aabadbb, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.450024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.471618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:58.475296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 816/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:58 2022] PUT /identity/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2/users/ae0afd5cae1b4bb5a4a6cc058aabadbb/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:58.479007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:58.480390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.542127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:58.620623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f77b-2217-42ff-9f79-bb622a9962f2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.633808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.634773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.635068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.635339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.636266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.637591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.661921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 817/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:36:58.666603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.684011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 812/1629] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 12 15:36:58.689590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc2245-a5b1-4e30-a686-12048a620d8b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.698552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.699011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.699295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.699538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:58.699658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] PATH_INFO: `/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2/users/ae0afd5cae1b4bb5a4a6cc058aabadbb/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.712085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc75273e8bd74a7a896ff0e9feb781e2, user_id=ae0afd5cae1b4bb5a4a6cc058aabadbb, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:58.714660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0557c-f689-4756-9a20-00a5fb7707b6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:58.759237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 818/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:36:58 2022] PUT /identity/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2/users/ae0afd5cae1b4bb5a4a6cc058aabadbb/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:36:58.761545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.761869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.762218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.763197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:58.808493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3bcd5-1553-4e7b-b943-601e7cb6e7eb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:58.824498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] MFA Rules not processed for user `ae0afd5cae1b4bb5a4a6cc058aabadbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:58.825785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:58.894560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:58.895334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:58.895626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:58.895907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:58.980598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 819/1631] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:36:58.991461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:59.006831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d435a-8dcc-41f9-860a-1f024955980a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:59.013136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:59.014190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:59.015340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:59.015839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:59.032128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:59.033667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b70ad-b9e4-4e2f-811a-3ac7f1da179b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:59.038567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:59.041264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f68a68-74c8-4225-a0b5-c120cd90898d service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:59.071881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 820/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:36:59.076516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 813/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:36:59.089367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431433-a7c3-4156-afbf-a6ae5103a801 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:59.090091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431433-a7c3-4156-afbf-a6ae5103a801 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:59.090472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431433-a7c3-4156-afbf-a6ae5103a801 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:59.092387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 821/1634] 10.222.0.20 () {54 vars in 912 bytes} [Mon Sep 12 15:36:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:36:59.102110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:59.102404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:59.103165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:59.159182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:36:59.161039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:36:59.185151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 814/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:36:59.234080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4431433-a7c3-4156-afbf-a6ae5103a801 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:59.248223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4431433-a7c3-4156-afbf-a6ae5103a801 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:59.257182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:59.258211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:59.258727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:59.259270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:59.274488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:59.277400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:59.309953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 822/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:36:59.634816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:36:59.796856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ee4ff-6102-4186-8be4-0efc55230c99 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:36:59.806829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:36:59.807554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:36:59.807848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:36:59.808129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] PATH_INFO: `/v3/users/bd428de0e4a54621bd6d55309ada03af` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:36:59.809946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd428de0e4a54621bd6d55309ada03af)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:36:59.814085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:36:59.814085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 12 15:36:59.814085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: Sep 12 15:36:59.814085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:36:59.885894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:36:59.954992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd428de0e4a54621bd6d55309ada03af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:36:59.956002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd428de0e4a54621bd6d55309ada03af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:36:59.956895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd428de0e4a54621bd6d55309ada03af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:36:59.985572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd428de0e4a54621bd6d55309ada03af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.031051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2f9967c1-099b-491d-b4d0-6397b115f13c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd428de0e4a54621bd6d55309ada03af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.050048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:00.068979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 815/1637] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:36:59 2022] DELETE /identity/v3/users/bd428de0e4a54621bd6d55309ada03af => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:00.073423 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258d5b-1736-42bc-bde2-df0cfc64c499 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:00.075522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee344-8437-488a-9e7b-d42d58234d12 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:00.076525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee344-8437-488a-9e7b-d42d58234d12 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:00.076525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ee344-8437-488a-9e7b-d42d58234d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:00.130774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-266ee344-8437-488a-9e7b-d42d58234d12 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:00.132029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-266ee344-8437-488a-9e7b-d42d58234d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:00.236991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:00.238628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:00.238829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:00.239134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] PATH_INFO: `/v3/users/66e3d6e996a54f5a9bc08a3dbfefd1a7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:00.242379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=66e3d6e996a54f5a9bc08a3dbfefd1a7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:00.248305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:00.261974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:00.285093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 816/1638] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:00.348799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66e3d6e996a54f5a9bc08a3dbfefd1a7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.349920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66e3d6e996a54f5a9bc08a3dbfefd1a7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.352368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66e3d6e996a54f5a9bc08a3dbfefd1a7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.370772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66e3d6e996a54f5a9bc08a3dbfefd1a7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.402802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66e3d6e996a54f5a9bc08a3dbfefd1a7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:00.431975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 823/1639] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:00 2022] DELETE /identity/v3/users/66e3d6e996a54f5a9bc08a3dbfefd1a7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:00.436313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:00.437159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:00.437449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:00.478373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:00.480204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:00.486655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:00.487644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:00.635147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 817/1640] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:00.635375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60d01-2ff4-4029-8490-a3104a568a1b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:00.639951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:00.640637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:00.640964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:00.641427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:00.643597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:00.658037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:00.659563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:00.673991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 824/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:00.792382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:00.812943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ee0ea-a7e4-41e2-8b64-f23efdec3361 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:00.818251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:00.818979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:00.819449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:00.819842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:00.838326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:00.840001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:00.858748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 818/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:02.513548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:02.516199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:02.530129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:02.532250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:02.554161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb8958-0089-4b68-80a0-8ca55600ba4c service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:02.560814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bdfe5-d5bd-4b75-87d8-42aad8d7abb8 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:02.569278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:02.579996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:02.580574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:02.581605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] PATH_INFO: `/v3/projects/fb0fd295845f4e7a9d5f3eb5f1073085` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:02.585745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb0fd295845f4e7a9d5f3eb5f1073085)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:02.591562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 15:37:02.591562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 12 15:37:02.591562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: Sep 12 15:37:02.594666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:02.598173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:02.598828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:02.599113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:02.599387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] PATH_INFO: `/v3/projects/71cff1e605f14117800fa9ee36eb4f1e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:02.602240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71cff1e605f14117800fa9ee36eb4f1e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:02.607387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:02.672472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:02.739112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71cff1e605f14117800fa9ee36eb4f1e', 'request_id': 'req-ff17243c-57f9-4013-bc0e-fc6649630383'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.752584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb0fd295845f4e7a9d5f3eb5f1073085', 'request_id': 'req-309f4860-f2de-45c6-9273-7ea936649b30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.754729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71cff1e605f14117800fa9ee36eb4f1e', 'request_id': 'req-ff17243c-57f9-4013-bc0e-fc6649630383'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.768589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb0fd295845f4e7a9d5f3eb5f1073085', 'request_id': 'req-309f4860-f2de-45c6-9273-7ea936649b30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.768790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71cff1e605f14117800fa9ee36eb4f1e', 'request_id': 'req-ff17243c-57f9-4013-bc0e-fc6649630383'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.796599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] The token cache is being invalidate because project 71cff1e605f14117800fa9ee36eb4f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:02.797087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.798776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.802110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb0fd295845f4e7a9d5f3eb5f1073085', 'request_id': 'req-309f4860-f2de-45c6-9273-7ea936649b30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.802355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 819/1643] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:02 2022] DELETE /identity/v3/projects/71cff1e605f14117800fa9ee36eb4f1e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:02.806500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:02.809119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:02.823013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] The token cache is being invalidate because project fb0fd295845f4e7a9d5f3eb5f1073085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:02.823981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.823981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-309f4860-f2de-45c6-9273-7ea936649b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:02.826645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 825/1644] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:02 2022] DELETE /identity/v3/projects/fb0fd295845f4e7a9d5f3eb5f1073085 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:02.834508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a3a5-afcb-46dc-874b-56e296e1f2f7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:02.834972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a3a5-afcb-46dc-874b-56e296e1f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:02.837328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a3a5-afcb-46dc-874b-56e296e1f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:02.944685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-84d7a3a5-afcb-46dc-874b-56e296e1f2f7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:02.951513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-84d7a3a5-afcb-46dc-874b-56e296e1f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:02.989127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17243c-57f9-4013-bc0e-fc6649630383 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:02.993629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 826/1645] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:02.996091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:02.996384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:02.996659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.000415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:03.001285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.003458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.004729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] PATH_INFO: `/v3/users/ae0afd5cae1b4bb5a4a6cc058aabadbb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.007656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0afd5cae1b4bb5a4a6cc058aabadbb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:03.014141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:03.037968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:03.067616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:03.069084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.099192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 827/1646] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:03.103126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:03.109228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.123736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9bab4-3994-481b-ba88-28fa232b1725 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:03.131018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:03.131684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.131972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.132250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] PATH_INFO: `/v3/users/a14a86493cb3425b977f1f660b05a905` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.133728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14a86493cb3425b977f1f660b05a905)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:03.138010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:03.145587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0afd5cae1b4bb5a4a6cc058aabadbb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.149633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0afd5cae1b4bb5a4a6cc058aabadbb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.154192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0afd5cae1b4bb5a4a6cc058aabadbb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.172467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0afd5cae1b4bb5a4a6cc058aabadbb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.224005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0afd5cae1b4bb5a4a6cc058aabadbb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.242126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14a86493cb3425b977f1f660b05a905'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.243161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14a86493cb3425b977f1f660b05a905'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.243358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14a86493cb3425b977f1f660b05a905'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.261251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 820/1647] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:02 2022] DELETE /identity/v3/users/ae0afd5cae1b4bb5a4a6cc058aabadbb => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:03.265531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:03.267142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.282561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14a86493cb3425b977f1f660b05a905'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.303554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1823b13f-2126-4498-a305-26e1ae3cd933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14a86493cb3425b977f1f660b05a905'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:03.335938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:03.351692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 828/1648] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:03 2022] DELETE /identity/v3/users/a14a86493cb3425b977f1f660b05a905 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:03.372947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.373595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.374698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.421179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629042ce-7e76-4efd-b483-227f901497f9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:03.430344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:03.433851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.536162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:03.537486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.538195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.538488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.539545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:03.539866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:03.542336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:03.561570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 821/1649] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:37:03.566313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:03.568912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.586795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 829/1650] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:03.594437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097f9f0-432d-48ba-a661-90a21aa39b57 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:03.599496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:03.600125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.600468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.600835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.602161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:03.604510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:03.612597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:03.615098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:03.740655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 822/1651] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:03 2022] POST /identity/v3/projects => generated 405 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:03.745550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:03.752065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce336-fea9-4864-9d9a-18b469ffffaf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:03.794938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:03.871392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:03.872302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:03.872729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:03.873222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:03.987680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9a36-ae6f-40f2-a62c-aad877abcf73 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.001456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:04.002772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.003069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.003347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.003995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.006775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.019715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.021415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.039906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 830/1652] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:04.076606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 823/1653] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:03 2022] POST /identity/v3/users => generated 303 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:04.081785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.094980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027688-f55c-4a31-8071-a7469489cbd6 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.108562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:04.110956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.112366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.113853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.114992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.117806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.138523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 831/1654] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:04.144535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.158148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e157f45-b867-4d53-aee4-d37a0bb9ad8e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.164776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:04.165513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.165835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.166116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.166842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.168573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761c58a-5d66-476a-9ac0-2e8ee4abeab6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.182455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 824/1655] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:04.189398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.207575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546b4d9-0c5c-45d0-a269-e8568be657f1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.213249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:04.215551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.215848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.216123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] PATH_INFO: `/v3/projects/9f14690e1966416eb9bf2533c3887b23/users/a9ddbe9057fa436b83fd729008fc8422/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.256480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f14690e1966416eb9bf2533c3887b23, user_id=a9ddbe9057fa436b83fd729008fc8422, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.261517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.292407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 832/1656] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:04 2022] PUT /identity/v3/projects/9f14690e1966416eb9bf2533c3887b23/users/a9ddbe9057fa436b83fd729008fc8422/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:04.304231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.304527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.304810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.369024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] MFA Rules not processed for user `a9ddbe9057fa436b83fd729008fc8422`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:04.376391 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:04.403958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:04.580746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 825/1657] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:04.585268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.696474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.698388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:04.773523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb480d80-87d4-404a-9fcf-fef58bc7b799 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.791919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:04.792704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:04.793092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:04.793475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:04.794261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:04.796365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:04.854625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 833/1658] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:04 2022] POST /identity/v3/projects => generated 403 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:04.858750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:04.874611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3e8e4-8947-43b5-bdeb-0ff8ae877efd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:04.909985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:05.007524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.008812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.009149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.009434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] PATH_INFO: `/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.011144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc75273e8bd74a7a896ff0e9feb781e2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.016016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.024417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598f948-eae6-4470-b306-f7fab3e31353 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.034678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.035415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.035804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.036182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.037140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.039028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.095828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 834/1659] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:04 2022] POST /identity/v3/users => generated 308 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:05.098899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.113385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b30b6-4dda-4407-8bfb-b4a90cb02468 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.137026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc75273e8bd74a7a896ff0e9feb781e2', 'request_id': 'req-796eff6b-96a2-4fe5-a027-d0649112b0f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:05.160114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc75273e8bd74a7a896ff0e9feb781e2', 'request_id': 'req-796eff6b-96a2-4fe5-a027-d0649112b0f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:05.182178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc75273e8bd74a7a896ff0e9feb781e2', 'request_id': 'req-796eff6b-96a2-4fe5-a027-d0649112b0f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:05.184594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:05.208695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] The token cache is being invalidate because project bc75273e8bd74a7a896ff0e9feb781e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:05.209456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:05.210411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-796eff6b-96a2-4fe5-a027-d0649112b0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:05.215114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 826/1660] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:04 2022] DELETE /identity/v3/projects/bc75273e8bd74a7a896ff0e9feb781e2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:05.222739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e40e03-ec5c-40dc-a25e-2e55c743170b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.228876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e40e03-ec5c-40dc-a25e-2e55c743170b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.228876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e40e03-ec5c-40dc-a25e-2e55c743170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.257660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.258324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.258606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.258920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.259544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.261218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.271123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-41e40e03-ec5c-40dc-a25e-2e55c743170b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:05.271979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-41e40e03-ec5c-40dc-a25e-2e55c743170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:05.293737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 835/1661] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:37:05.296503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.296716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 827/1662] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:05.305269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.305560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.305877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.306170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:05.337965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b68d-d943-44b3-bbbd-83f8856be915 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.354960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:05.355878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.356614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.356882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.357160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] PATH_INFO: `/v3/projects/594c3378a49f439595ea0a394d3c5c31/users/3bee9547c35b4bd0b5d7c73b8f715b35/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.363585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:05.389874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 828/1663] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:05.394488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.395246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594c3378a49f439595ea0a394d3c5c31, user_id=3bee9547c35b4bd0b5d7c73b8f715b35, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.399323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.411625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbfc0d-47a3-4b42-984e-faea6e91fd74 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.422087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.423058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.423518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.423977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.424818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:05.427356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.433754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.433931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 836/1664] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:05 2022] PUT /identity/v3/projects/594c3378a49f439595ea0a394d3c5c31/users/3bee9547c35b4bd0b5d7c73b8f715b35/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:05.439232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.455222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 829/1665] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:37:05.464716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.486738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba6260-0a90-4a0c-9828-8b2deba42f20 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.627495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f9bd3-7f9d-451d-bda6-765c58383e82 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.638691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.639973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.640475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.640951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.641847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.643438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.644076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.644358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.644681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.645606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.647249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.647656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.668845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 837/1666] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:05.674105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.699494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cd36-c306-49b6-8369-e05492f23c46 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.710981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.711668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.711960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.712244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] PATH_INFO: `/v3/projects/594c3378a49f439595ea0a394d3c5c31/users/3bee9547c35b4bd0b5d7c73b8f715b35/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.721353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 830/1667] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:05 2022] POST /identity/v3/projects => generated 377 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:05.721511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594c3378a49f439595ea0a394d3c5c31, user_id=3bee9547c35b4bd0b5d7c73b8f715b35, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.723116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:05.723592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee33cc-59b2-413d-a5ab-083e35f603f8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:05.759621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 838/1668] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:05 2022] PUT /identity/v3/projects/594c3378a49f439595ea0a394d3c5c31/users/3bee9547c35b4bd0b5d7c73b8f715b35/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:05.761890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.762009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.762291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.815678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:05.843339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] MFA Rules not processed for user `3bee9547c35b4bd0b5d7c73b8f715b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:05.847617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:05.850563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902e0f-d2dc-4ceb-b3b8-457b183c800e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:05.975157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:05.986823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:05.987560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:05.988306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:05.988306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:05.988910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:05.993613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.035899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 839/1669] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:05 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:06.047204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.047629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 831/1670] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:05 2022] POST /identity/v3/users => generated 289 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:06.052620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.062508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:06.072275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f89b88-959a-4d6f-9f38-121b60810b39 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.080876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.081923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.082226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.082503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.083228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.090681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.178506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 840/1671] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:06.181005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.182591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d61cec-af94-4d27-8cda-821093ee2aa7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.194481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.195138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.195699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.196204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.203358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ece6e-1f55-4e23-b27f-34d366b44c46 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.209618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.210279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.210566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.210848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.211506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.213817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.214230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.216675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c73e0-8d6a-4d6a-b91b-3a6f2bd1eb39 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.230417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 841/1672] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:06.232043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.250027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 832/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:06 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:06.253057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909cacd-ffd8-46e0-92d5-4c75bc336d15 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.259483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.260181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.260467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.260744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] PATH_INFO: `/v3/projects/3b6b444e93e44ca382e3116061c33a81/users/8ad0d0b6886d4d16b2cb2a5165be763b/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.304408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6b444e93e44ca382e3116061c33a81, user_id=8ad0d0b6886d4d16b2cb2a5165be763b, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.307064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.340678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 842/1674] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:06 2022] PUT /identity/v3/projects/3b6b444e93e44ca382e3116061c33a81/users/8ad0d0b6886d4d16b2cb2a5165be763b/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:06.351319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.351943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.352434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.451604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] MFA Rules not processed for user `8ad0d0b6886d4d16b2cb2a5165be763b`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:06.453382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:06.604520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.658385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 833/1675] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:06.660435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.740491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024afce-8d57-4f50-9bf4-baa23783da0d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.752496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.754988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.755436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.756849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] PATH_INFO: `/v3/users/a9ddbe9057fa436b83fd729008fc8422` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.766488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ddbe9057fa436b83fd729008fc8422)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.774232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.832230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdda703-5dac-4d59-b889-7b57b89489e5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:06.847022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:06.847812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:06.848208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:06.848585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:06.849402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:06.855060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:06.929306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 834/1676] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:06 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:06.930432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:06.934225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:06.948638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:06.950136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ddbe9057fa436b83fd729008fc8422'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:06.952954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ddbe9057fa436b83fd729008fc8422'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:06.955023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ddbe9057fa436b83fd729008fc8422'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:06.963114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:06.993866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ddbe9057fa436b83fd729008fc8422'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.042368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ddbe9057fa436b83fd729008fc8422'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.083759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 843/1677] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:06 2022] DELETE /identity/v3/users/a9ddbe9057fa436b83fd729008fc8422 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:07.087075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.089649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.152435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272040ed-ebfa-48a2-a61e-b3dee8b9c3c9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:07.307390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496318c5-7d47-4b84-b498-9e140d9643b3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:07.329911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:07.330578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.330876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.331135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] PATH_INFO: `/v3/projects/9a2f7a7e414840b6a8745b8c6f355e4f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.332429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2f7a7e414840b6a8745b8c6f355e4f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:07.334235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:07.338648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:07.341349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.342537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.342829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.343488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:07.346067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abfdbe-79a5-488a-b12d-0c221463ebf7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:07.414884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 844/1678] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:07 2022] POST /identity/v3/users => generated 295 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:07.417527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874fb1-55cb-423e-9e9d-b27a415d44df None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.418014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874fb1-55cb-423e-9e9d-b27a415d44df None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.418424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68874fb1-55cb-423e-9e9d-b27a415d44df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.440800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-68874fb1-55cb-423e-9e9d-b27a415d44df None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:07.442044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-68874fb1-55cb-423e-9e9d-b27a415d44df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.442483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2f7a7e414840b6a8745b8c6f355e4f', 'request_id': 'req-fe1aec96-fe57-4588-9939-fedc0b832f65'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.457281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2f7a7e414840b6a8745b8c6f355e4f', 'request_id': 'req-fe1aec96-fe57-4588-9939-fedc0b832f65'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.470561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2f7a7e414840b6a8745b8c6f355e4f', 'request_id': 'req-fe1aec96-fe57-4588-9939-fedc0b832f65'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.497019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] The token cache is being invalidate because project 9a2f7a7e414840b6a8745b8c6f355e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:07.497167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.499697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:07.501791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 835/1679] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:06 2022] DELETE /identity/v3/projects/9a2f7a7e414840b6a8745b8c6f355e4f => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:07.502591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.505628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.619306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:07.624827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 845/1680] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:07.626615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.627467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.630205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.663529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aec96-fe57-4588-9939-fedc0b832f65 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:07.674969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:07.675581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:07.676562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.676850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.677130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.677819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:07.678289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.679459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565e19-42bb-493f-bd17-caeaa101e517 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:07.702547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 846/1681] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:07.703666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.706950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.710869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 836/1682] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:07.712172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.712464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.712742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.758953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:07.760501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:07.780957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 837/1683] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:07.782094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.808418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d394c8d-65ac-44b8-9ba7-4547ae0ccecc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:07.818094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:07.819029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.819669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.820092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] PATH_INFO: `/v3/projects/efc6fd4346eb430488a7a5690bfd62d2/users/77422b32b0ae4e5a97351d2c4afd56f1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.848502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6354ff-b5e0-4391-a5e7-ad9c71738317 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:07.852869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc6fd4346eb430488a7a5690bfd62d2, user_id=77422b32b0ae4e5a97351d2c4afd56f1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:07.860862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:07.862528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:07.863287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:07.863651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:07.864008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:07.882970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:07.887854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:07.890804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:07.892561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 838/1684] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:07 2022] PUT /identity/v3/projects/efc6fd4346eb430488a7a5690bfd62d2/users/77422b32b0ae4e5a97351d2c4afd56f1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:07.893412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.911024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 847/1685] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:07.912606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:07.932056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15368f00-6a43-46f7-8d52-e41e26a4a668 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.038857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33c03-9441-4fe1-a79d-0e52b01a07c1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.069565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.072946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.073687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.074072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.074547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.074646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.074922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.075301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.075598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.076364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:08.076977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.079602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.079799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.112684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 848/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:08.113676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.115513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 839/1687] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:37:08.120038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.137901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f601d5-77a3-4dc1-acf1-dbf4d45be7cc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.142318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc8ed3-18e2-4c26-8fb9-5be29ba7d6d6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.152760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.153456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.153761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.154047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] PATH_INFO: `/v3/projects/efc6fd4346eb430488a7a5690bfd62d2/users/77422b32b0ae4e5a97351d2c4afd56f1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.154562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.156057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.156786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.157252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.158198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.160048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc6fd4346eb430488a7a5690bfd62d2, user_id=77422b32b0ae4e5a97351d2c4afd56f1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.163767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.166528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee8dbf-f2f1-43fa-a03d-505e7cef8414 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.198084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 849/1688] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:08 2022] PUT /identity/v3/projects/efc6fd4346eb430488a7a5690bfd62d2/users/77422b32b0ae4e5a97351d2c4afd56f1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:08.212614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.213407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.213407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.251493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 840/1689] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:08 2022] POST /identity/v3/projects => generated 405 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:08.252512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.266416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] MFA Rules not processed for user `77422b32b0ae4e5a97351d2c4afd56f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:08.267976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:08.402228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a676302-702c-4ca0-8c87-6ded9fbbabdb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.413125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.414687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.415354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.415959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.416871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.422353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 850/1690] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:08.428084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.429868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.431561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:08.492952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 841/1691] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:08 2022] POST /identity/v3/users => generated 303 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:08.496545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.522334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75b033-87eb-48e8-9521-62f7efe1da86 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.531812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.533007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.533347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.533783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.534635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.537982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.546841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:08.559377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 842/1692] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:08.560317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.574003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09059a8a-8945-42a7-b82c-a84197317c0d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.581366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.581589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.581888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.582166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.582998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.585193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.586537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5683b-13b5-468a-a9e7-d38f05fb895e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.600917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.601494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.601808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.602245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.604127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:08.607077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 843/1693] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:08.610880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:08.625596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f432-a096-4490-b8ab-0e9ccce4a1d3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:08.635511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:08.636132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:08.636623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.637524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.637938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] PATH_INFO: `/v3/projects/01f08259f1434709b094323c3225a7f8/users/f2dcaae30abc4b68997e7d34e770c2a4/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.664000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f08259f1434709b094323c3225a7f8, user_id=f2dcaae30abc4b68997e7d34e770c2a4, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.666965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e77d3-bf1b-4b0a-8e81-a557bc5abc4d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.701514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 844/1694] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:08 2022] PUT /identity/v3/projects/01f08259f1434709b094323c3225a7f8/users/f2dcaae30abc4b68997e7d34e770c2a4/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:08.705142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:08.705506 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:08.706750 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:08.756182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:08.758284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:08.781773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 851/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:08.783534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:08.787393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:08.964880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 845/1696] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:08.989382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:09.111262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad9213-8dd5-4624-89db-c55953665518 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:09.197784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:09.198721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:09.199145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:09.199561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:09.230244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:09.232290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:09.258712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 852/1697] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:09.828031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:09.829761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:09.852732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac932e-629b-4a0b-9302-53c143a5fcca None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:09.866814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:09.867738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:09.869190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:09.869853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] PATH_INFO: `/v3/projects/9f14690e1966416eb9bf2533c3887b23` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:09.875090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f14690e1966416eb9bf2533c3887b23)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:09.878627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:10.037653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f14690e1966416eb9bf2533c3887b23', 'request_id': 'req-eb26b799-482f-408a-9bd3-990bf4a241a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.051098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f14690e1966416eb9bf2533c3887b23', 'request_id': 'req-eb26b799-482f-408a-9bd3-990bf4a241a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.075210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f14690e1966416eb9bf2533c3887b23', 'request_id': 'req-eb26b799-482f-408a-9bd3-990bf4a241a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.095856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] The token cache is being invalidate because project 9f14690e1966416eb9bf2533c3887b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:10.096193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.106289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-eb26b799-482f-408a-9bd3-990bf4a241a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.108848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 846/1698] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:09 2022] DELETE /identity/v3/projects/9f14690e1966416eb9bf2533c3887b23 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:10.114668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:10.132347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b258c-7a0b-4fc5-b4c7-43fe94369392 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:10.267307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:10.268295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:10.268782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:10.269279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] PATH_INFO: `/v3/users/3bee9547c35b4bd0b5d7c73b8f715b35` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:10.270861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bee9547c35b4bd0b5d7c73b8f715b35)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:10.273141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:10.313865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:10.382935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bee9547c35b4bd0b5d7c73b8f715b35'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.386748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bee9547c35b4bd0b5d7c73b8f715b35'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.387700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bee9547c35b4bd0b5d7c73b8f715b35'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.406254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bee9547c35b4bd0b5d7c73b8f715b35'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.434779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bee9547c35b4bd0b5d7c73b8f715b35'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:10.466260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 853/1699] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:10 2022] DELETE /identity/v3/users/3bee9547c35b4bd0b5d7c73b8f715b35 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:11.887558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e4d8-6d36-49b8-ac60-d941e680d423 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:11.888076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e4d8-6d36-49b8-ac60-d941e680d423 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:11.888147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e4d8-6d36-49b8-ac60-d941e680d423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:11.963853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-44c0e4d8-6d36-49b8-ac60-d941e680d423 None None] MFA Rules not processed for user `5d1ab08e56ca4ad692869347a8cd7e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:11.966225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-44c0e4d8-6d36-49b8-ac60-d941e680d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:12.036652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:12.167571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 847/1700] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:12.179384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:12.181062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:12.393504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e31f1-bcca-4d20-a732-9b0fbd07b527 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:12.417536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:12.418706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:12.419426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:12.420590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:12.456168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:12.459003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:12.475131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 854/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:12.949554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:12.950100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:12.950336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.049382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] MFA Rules not processed for user `f2dcaae30abc4b68997e7d34e770c2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:13.051772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.180785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.183256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.255971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 848/1702] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:13.260661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.261950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.349565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5283c-3957-4a7e-9673-aad86fdae8ef service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.362952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.363872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.364304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.364724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] PATH_INFO: `/v3/projects/594c3378a49f439595ea0a394d3c5c31` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.368459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594c3378a49f439595ea0a394d3c5c31)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.370896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.434801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:13.443530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b54c97-5ef1-4c24-ad98-69885fa1ce7b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.458569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.461579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.461964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.462373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.465220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.468141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.489997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594c3378a49f439595ea0a394d3c5c31', 'request_id': 'req-634fe6ce-9258-4043-a0cc-5a11be0dc196'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:13.501795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594c3378a49f439595ea0a394d3c5c31', 'request_id': 'req-634fe6ce-9258-4043-a0cc-5a11be0dc196'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:13.514463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 849/1703] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:13 2022] POST /identity/v3/projects => generated 403 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:13.519047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594c3378a49f439595ea0a394d3c5c31', 'request_id': 'req-634fe6ce-9258-4043-a0cc-5a11be0dc196'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:13.519945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.542710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] The token cache is being invalidate because project 594c3378a49f439595ea0a394d3c5c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:13.543094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:13.543837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-634fe6ce-9258-4043-a0cc-5a11be0dc196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:13.546434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 855/1704] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:13 2022] DELETE /identity/v3/projects/594c3378a49f439595ea0a394d3c5c31 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:13.551896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605540c-99fd-4cea-8810-c79651b04741 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.552271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605540c-99fd-4cea-8810-c79651b04741 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.552725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605540c-99fd-4cea-8810-c79651b04741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.601042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0605540c-99fd-4cea-8810-c79651b04741 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:13.606840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0605540c-99fd-4cea-8810-c79651b04741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.688160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:13.726338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f205c29-7626-4607-980d-85b17ba336b2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.733473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.734708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.735139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.735594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.736950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.741450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.774149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 856/1705] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:13.783316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.783826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.784288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.797250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 850/1706] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:13 2022] POST /identity/v3/users => generated 308 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:13.801306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.803291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.844525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ee964-7991-4b0b-b860-8bbe11e4d7a8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.845949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:13.847681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:13.860127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.861311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.862654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.862944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.863619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.864177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 857/1707] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:13.869804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.873675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.881511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2414092-7d52-44dd-bd44-96d61fcb656c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.888079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.891507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.891991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.892417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.893398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:13.893880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.895668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.917312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 851/1708] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:13.920543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.921131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 858/1709] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:37:13.925049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:13.940377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f3b6-e139-470c-ad6e-3eb5e8a66913 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.947414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.948160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.949339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.950379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:13.950961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:13.953998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:13.963670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525035-0001-4fdd-bc39-9e71f21e9c01 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:13.980685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:13.984497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:13.984944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:13.986334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] PATH_INFO: `/v3/projects/689e8a3afb754e54a43fed82c5f4f594/users/5e6bbadad6eb4281b36de161b5ae1375/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.028640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 859/1710] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:14.031583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=689e8a3afb754e54a43fed82c5f4f594, user_id=5e6bbadad6eb4281b36de161b5ae1375, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.032847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.041065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.074550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 852/1711] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:13 2022] PUT /identity/v3/projects/689e8a3afb754e54a43fed82c5f4f594/users/5e6bbadad6eb4281b36de161b5ae1375/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:14.082856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.150675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e11b0-1d80-4a36-b031-db03551ee13c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.196372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:14.205139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b4e9-1281-488e-9f32-bcd80f1e5b8a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.223403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.224819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.225072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.225458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.226294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.234324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.247010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 853/1712] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:14.251086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.297467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4685d-dd72-45cf-a025-9980a9be8a51 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.299930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.300924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.301466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.302036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.302992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.305527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.312842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.313665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.314226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.314609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] PATH_INFO: `/v3/projects/689e8a3afb754e54a43fed82c5f4f594/users/5e6bbadad6eb4281b36de161b5ae1375/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.317749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=689e8a3afb754e54a43fed82c5f4f594, user_id=5e6bbadad6eb4281b36de161b5ae1375, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.324841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.349600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 860/1713] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:14 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:14.354354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 854/1714] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:14 2022] PUT /identity/v3/projects/689e8a3afb754e54a43fed82c5f4f594/users/5e6bbadad6eb4281b36de161b5ae1375/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:14.354818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.386334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5f85-13f6-45ba-a21a-df976c58af31 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.551516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.552162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.552446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.552724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.553600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.556971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.584851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.590217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 861/1715] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:14.602969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467ba91-0394-40ca-8c9f-8eac48c160ff admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.616471 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.617548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.618034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.618439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.619237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.623018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f939d7c-6a74-4bbe-aa68-c0924e23d2d7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.655747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 855/1716] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:37:14.660579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:14.681221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebee02-6330-4e4a-a247-a82d98301177 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:14.709158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:14.709885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.713340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.713476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] PATH_INFO: `/v3/projects/c18510d59b5e42a08cbae34e622a9087/users/f1cc2e1b76cf423097ae0e8382001046/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.759241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18510d59b5e42a08cbae34e622a9087, user_id=f1cc2e1b76cf423097ae0e8382001046, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:14.764176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:14.795845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 862/1717] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:14 2022] PUT /identity/v3/projects/c18510d59b5e42a08cbae34e622a9087/users/f1cc2e1b76cf423097ae0e8382001046/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:14.808426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347580df-1912-4c4c-b8c7-860184b3fd8c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:14.809049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347580df-1912-4c4c-b8c7-860184b3fd8c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:14.809568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347580df-1912-4c4c-b8c7-860184b3fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:14.912871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-347580df-1912-4c4c-b8c7-860184b3fd8c None None] MFA Rules not processed for user `f1cc2e1b76cf423097ae0e8382001046`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:14.916164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-347580df-1912-4c4c-b8c7-860184b3fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:15.130106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 856/1718] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:15.134159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:15.225603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ca8e-1955-44f4-a1a1-4c143f51ab78 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.227348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ca8e-1955-44f4-a1a1-4c143f51ab78 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.227756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ca8e-1955-44f4-a1a1-4c143f51ab78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.269494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:15.333230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9577ca8e-1955-44f4-a1a1-4c143f51ab78 None None] MFA Rules not processed for user `d718e35636464834bc20a6eda0599cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:15.334812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9577ca8e-1955-44f4-a1a1-4c143f51ab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:15.342105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f301306-0f17-4eb0-8be5-addc764bb019 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:15.348536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:15.349665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.350228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.356255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.358713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:15.361173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:15.427265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 863/1719] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:15 2022] POST /identity/v3/projects => generated 371 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:15.432059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:15.445227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:15.593607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 857/1720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:15.595687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.595986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.596268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.632678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ea72-8ef4-433b-b1f9-43fc51d9e560 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:15.638197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] MFA Rules not processed for user `d718e35636464834bc20a6eda0599cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:15.641137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:15.646422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:15.647453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.648717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.653104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.655733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:15.670253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7a01b-ff6d-4be5-9627-6d9db1f2564d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:15.742730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 864/1721] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:15 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:15.743965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.744259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.744531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.862486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] MFA Rules not processed for user `1ca7bef620284345a977088f9a0d2a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:15.864571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:15.893192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 858/1722] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:15.893192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:15.938799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d56c2-3ae7-48ce-9aa0-14cfe21ef0a1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:15.948274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:15.950881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:15.951336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:15.952621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:15.954810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:15.956511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:15.981755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 859/1723] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:15.981755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:15.988336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:16.010863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 865/1724] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:16.010863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:16.010863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:16.252537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034637d-6cfe-4ff4-97bc-2575cc0fa72e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.253070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d6fce-1031-45f4-8cef-168a344eaf62 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.257951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.259701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.260001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.260274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.263282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:16.265467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.266603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.266953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.267186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.286072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:16.291531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:16.362947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 860/1725] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:16.364046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:16.390816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce43c1-2a0a-4cbf-98b1-020deb5aa4a0 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.406745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.407768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.408409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.409556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] PATH_INFO: `/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4/users/2c4943a1eed14ae3a51896d48b843596/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.437909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:16.466480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:16.484847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3246ceeb7d64de8b095f5d7c305b7c4, user_id=2c4943a1eed14ae3a51896d48b843596, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:16.490728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:16.554375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 861/1726] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:16 2022] PUT /identity/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4/users/2c4943a1eed14ae3a51896d48b843596/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:16.556036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:16.566259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:16.602747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:16.608975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:16.649324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 866/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:16.650370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:16.675961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24e14b-584f-4518-90a2-8979d5b4f8c4 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.783499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21602-7e32-4101-bf28-cca3d3f86a51 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.790102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.790956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.791312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.791902 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.822389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.823011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.823298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.823576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.824238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:16.825936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:16.856558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 867/1728] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:16.861690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:16.881152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a6b0-390e-41ce-bd5d-3d6678cf2807 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:16.900488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:16.901188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.901475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.901826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] PATH_INFO: `/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4/users/2c4943a1eed14ae3a51896d48b843596/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:16.907247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3246ceeb7d64de8b095f5d7c305b7c4, user_id=2c4943a1eed14ae3a51896d48b843596, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:16.910985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cec12-edde-421e-8cea-7db347ec6f44 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:16.971857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 868/1729] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:16 2022] PUT /identity/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4/users/2c4943a1eed14ae3a51896d48b843596/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:16.982707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:16.983781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:16.984731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:17.003496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:17.008153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:17.042909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 862/1730] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:17.067119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] MFA Rules not processed for user `2c4943a1eed14ae3a51896d48b843596`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:17.068507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:17.309424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 869/1731] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:17.321322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:17.324529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:17.479876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66990-5fa3-455c-ba3e-9db75a31f4cd service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:17.610961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:17.638136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:17.638817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:17.639654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:17.641969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:17.666791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:17.670704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461066-442f-4e3e-8ac8-f52dfe32c201 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:17.697661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 863/1732] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:17.914869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:17.932421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4ef69-82af-4223-aa29-bdb629846174 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:17.944252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:17.944895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:17.945179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:17.945735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.075791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:18.082897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3dedf-735c-43b5-872b-51b34b15cf83 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.083196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3dedf-735c-43b5-872b-51b34b15cf83 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.083476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3dedf-735c-43b5-872b-51b34b15cf83 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.088674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 864/1733] 10.222.0.20 () {54 vars in 912 bytes} [Mon Sep 12 15:37:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:37:18.099632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c932635-37ec-47ee-89b2-af27c18a662a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.100033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c932635-37ec-47ee-89b2-af27c18a662a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.100482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c932635-37ec-47ee-89b2-af27c18a662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.119001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:18.120626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:18.137281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 870/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:18.229862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7c932635-37ec-47ee-89b2-af27c18a662a None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:18.231524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7c932635-37ec-47ee-89b2-af27c18a662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:18.416709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 865/1735] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:18.525395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:18.526751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2f75-db0c-466c-bbd1-6c31dc197415 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.527970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2f75-db0c-466c-bbd1-6c31dc197415 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.529131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2f75-db0c-466c-bbd1-6c31dc197415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.529976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:18.578728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37ed2-e4ca-4830-9fff-3c000a1f43f0 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:18.593814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:18.594584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.594972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.596792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.614824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:18.616479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98215bf7-97fe-4de9-a632-22fcc2c3363a service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:18.635016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7d2e2f75-db0c-466c-bbd1-6c31dc197415 None None] MFA Rules not processed for user `a1a9b41251e146eba110fa1e2b1fc9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:18.636692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7d2e2f75-db0c-466c-bbd1-6c31dc197415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:18.642857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 871/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:18.832059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce828f-a30e-4885-8a1c-7bcd68a3f893 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.832293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 866/1737] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:18.832546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce828f-a30e-4885-8a1c-7bcd68a3f893 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.832927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce828f-a30e-4885-8a1c-7bcd68a3f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.833796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:18.834643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:18.834643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:18.880428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] MFA Rules not processed for user `a1a9b41251e146eba110fa1e2b1fc9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:18.888879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:18.893809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9fce828f-a30e-4885-8a1c-7bcd68a3f893 None None] MFA Rules not processed for user `5e6bbadad6eb4281b36de161b5ae1375`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:18.904979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9fce828f-a30e-4885-8a1c-7bcd68a3f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:18.908349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 867/1738] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:18.926731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:19.050792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 872/1739] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:19.119707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:19.126889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57c09f-9006-40dd-a5b0-99869a5a30a9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:19.139900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:19.140875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:19.141457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:19.142057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:19.153548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:19.156340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:19.187543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 868/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:19.514378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f0f8d-092d-4c62-b54f-8cb9c22b9322 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:19.514703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f0f8d-092d-4c62-b54f-8cb9c22b9322 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:19.514980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f0f8d-092d-4c62-b54f-8cb9c22b9322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:19.566947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca9f0f8d-092d-4c62-b54f-8cb9c22b9322 None None] MFA Rules not processed for user `5d1ab08e56ca4ad692869347a8cd7e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:19.568446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca9f0f8d-092d-4c62-b54f-8cb9c22b9322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:19.756034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 873/1741] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:20.173852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:20.175308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:20.391929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6e481-f289-4535-b71a-6f32a088bfb0 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:20.414796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:20.415443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.415732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.416018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.435197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:20.462124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:20.464385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08677956-659a-4f0e-b368-03fd38adb8e0 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:20.493366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 869/1742] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:20.579847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc147d3-8d6c-404b-becf-5db6d4c5c305 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.580878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc147d3-8d6c-404b-becf-5db6d4c5c305 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.581169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc147d3-8d6c-404b-becf-5db6d4c5c305 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.583308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 874/1743] 10.222.0.20 () {54 vars in 912 bytes} [Mon Sep 12 15:37:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:37:20.606098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed7c0-0f1b-4f57-8408-cb856a5cb524 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.606393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed7c0-0f1b-4f57-8408-cb856a5cb524 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.606655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eed7c0-0f1b-4f57-8408-cb856a5cb524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.645193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-69eed7c0-0f1b-4f57-8408-cb856a5cb524 None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:20.646508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-69eed7c0-0f1b-4f57-8408-cb856a5cb524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:20.663154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 870/1744] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:20.818927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc147d3-8d6c-404b-becf-5db6d4c5c305 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:20.835250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc147d3-8d6c-404b-becf-5db6d4c5c305 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:20.847578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:20.848267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.848559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.848839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.865420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:20.869319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f245e1-8c33-4411-ada1-05b23f53ca7b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:20.888488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 875/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:20.931828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8da478-efd8-406d-8cb4-61688541439e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.932206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8da478-efd8-406d-8cb4-61688541439e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.932507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8da478-efd8-406d-8cb4-61688541439e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.934419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 871/1746] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:37:20.946360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381337d8-9dbf-41f0-a7b4-36319932278c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:20.946868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381337d8-9dbf-41f0-a7b4-36319932278c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:20.947333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381337d8-9dbf-41f0-a7b4-36319932278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:20.989662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-381337d8-9dbf-41f0-a7b4-36319932278c None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:20.994300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-381337d8-9dbf-41f0-a7b4-36319932278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:21.022973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 876/1747] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:37:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:21.068780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c0be-1cc0-4644-a5b7-c1137bfe9763 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:21.069063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c0be-1cc0-4644-a5b7-c1137bfe9763 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:21.069503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5c0be-1cc0-4644-a5b7-c1137bfe9763 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:21.071379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 872/1748] 10.222.0.20 () {54 vars in 961 bytes} [Mon Sep 12 15:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:37:21.087237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b763da-54f9-4e0c-9a96-bdc93bfd4837 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:21.087530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b763da-54f9-4e0c-9a96-bdc93bfd4837 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:21.087884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b763da-54f9-4e0c-9a96-bdc93bfd4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:21.143016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a6b763da-54f9-4e0c-9a96-bdc93bfd4837 None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:21.144547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a6b763da-54f9-4e0c-9a96-bdc93bfd4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:21.166151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 877/1749] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Sep 12 15:37:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:21.166683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5c0be-1cc0-4644-a5b7-c1137bfe9763 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:21.189076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5c0be-1cc0-4644-a5b7-c1137bfe9763 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:21.204764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:21.205540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:21.205932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:21.206289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:21.227626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:21.229257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf9fd5-6191-4df2-8acf-04dfe8c4f217 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:21.267917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 873/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:21.499037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:21.499805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:21.500284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:21.648786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] MFA Rules not processed for user `930cbe41fc0741cfa5fbd81645181418`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:21.656749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:21.766928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:21.932622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 878/1751] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:22.444054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:22.444635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:22.444635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:22.529591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:22.550941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] MFA Rules not processed for user `d718e35636464834bc20a6eda0599cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:22.559395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:22.684122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57717e0c-ad2d-4e1f-abbf-daeafa58b82b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:22.699018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:22.702985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:22.703524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:22.704088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] PATH_INFO: `/v3/users/f2dcaae30abc4b68997e7d34e770c2a4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:22.707860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2dcaae30abc4b68997e7d34e770c2a4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:22.713157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:22.781433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 874/1752] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:22.839621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:22.863680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2dcaae30abc4b68997e7d34e770c2a4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:22.864746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2dcaae30abc4b68997e7d34e770c2a4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:22.865997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2dcaae30abc4b68997e7d34e770c2a4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:22.882110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7e1b-15ed-4235-8ead-eb820cbc79b6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:22.891516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2dcaae30abc4b68997e7d34e770c2a4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:22.903966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:22.904996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:22.905439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:22.906722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:22.912622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:22.922984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2dcaae30abc4b68997e7d34e770c2a4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:22.953091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 879/1753] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:22 2022] DELETE /identity/v3/users/f2dcaae30abc4b68997e7d34e770c2a4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:22.955641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:22.975817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:23.024940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 875/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:23.447473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:23.449075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:23.564331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:23.632058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ca226-9809-434c-97cd-5e7569faaf2f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:23.650472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:23.651361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:23.651844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:23.652266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] PATH_INFO: `/v3/projects/01f08259f1434709b094323c3225a7f8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:23.668376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f08259f1434709b094323c3225a7f8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:23.673416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:23.821295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f08259f1434709b094323c3225a7f8', 'request_id': 'req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:23.844313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f08259f1434709b094323c3225a7f8', 'request_id': 'req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:23.857350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f08259f1434709b094323c3225a7f8', 'request_id': 'req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:23.900505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] The token cache is being invalidate because project 01f08259f1434709b094323c3225a7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:23.900803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:23.903776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:23.907580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 880/1755] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:23 2022] DELETE /identity/v3/projects/01f08259f1434709b094323c3225a7f8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:23.909967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:23.924937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d5204-224e-4d99-93b2-97ac7b68a7dc service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:24.158515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:24.198001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:24.200086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:24.204773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:24.205232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] PATH_INFO: `/v3/users/5e6bbadad6eb4281b36de161b5ae1375` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:24.209097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6bbadad6eb4281b36de161b5ae1375)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:24.218432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:24.323040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6bbadad6eb4281b36de161b5ae1375'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:24.324721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6bbadad6eb4281b36de161b5ae1375'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:24.325485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6bbadad6eb4281b36de161b5ae1375'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:24.335725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6bbadad6eb4281b36de161b5ae1375'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:24.374244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0457c735-7d33-42cd-9cc3-f4d2442c13e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6bbadad6eb4281b36de161b5ae1375'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:24.397847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 876/1756] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:23 2022] DELETE /identity/v3/users/5e6bbadad6eb4281b36de161b5ae1375 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:24.876088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:24.877927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:24.903509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0b1d8-9b6f-4de2-ba5e-447be0965e0b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:25.054116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1794e5-6ad9-486b-b418-07049517bece None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.054803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1794e5-6ad9-486b-b418-07049517bece None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.054803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1794e5-6ad9-486b-b418-07049517bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.087176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1d1794e5-6ad9-486b-b418-07049517bece None None] MFA Rules not processed for user `1ca7bef620284345a977088f9a0d2a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:25.088486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1d1794e5-6ad9-486b-b418-07049517bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:25.101063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:25.101543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.101897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.103213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] PATH_INFO: `/v3/projects/689e8a3afb754e54a43fed82c5f4f594` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.103413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689e8a3afb754e54a43fed82c5f4f594)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:25.105576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:25.190874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 877/1757] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:25.243332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689e8a3afb754e54a43fed82c5f4f594', 'request_id': 'req-30dbc038-2f26-46ba-9c49-036e6fcdfad5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:25.266829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689e8a3afb754e54a43fed82c5f4f594', 'request_id': 'req-30dbc038-2f26-46ba-9c49-036e6fcdfad5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:25.285035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689e8a3afb754e54a43fed82c5f4f594', 'request_id': 'req-30dbc038-2f26-46ba-9c49-036e6fcdfad5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:25.312892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:25.318802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] The token cache is being invalidate because project 689e8a3afb754e54a43fed82c5f4f594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:25.319506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:25.324899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-30dbc038-2f26-46ba-9c49-036e6fcdfad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:25.330282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 881/1758] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:24 2022] DELETE /identity/v3/projects/689e8a3afb754e54a43fed82c5f4f594 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:25.332516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.332654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.333002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.381554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:25.386157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:25.581440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 878/1759] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:25.591529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.592969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.593508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.644233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:25.647310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:25.687696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 882/1760] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:25.691872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:25.711399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904cab-51db-4a41-9ea5-42e65e11d923 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:25.735605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:25.740497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.741682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.744732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.746985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:25.747888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:25.756137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:25.804108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 879/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:37:25.808201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:25.823589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec6b3-0de5-42c6-bb07-b826cdf2a39f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:25.836902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:25.837579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:25.837980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:25.838260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:25.840082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:25.846612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:25.936736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 883/1762] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:25 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:25.941212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:25.956358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:25.961500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840842c-7e7d-4b5c-a32d-ebc3e45a6513 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:26.151942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:26.153329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:26.154172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:26.155271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:26.156757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:26.161970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:26.203626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 880/1763] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:25 2022] POST /identity/v3/users => generated 284 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:26.208037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:26.238260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2396a-aa97-4615-af2b-a9cfebc01c4e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:26.267813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:26.269275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:26.269571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:26.269976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:26.270738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:26.275256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:26.298886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 884/1764] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:26.304316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:26.322056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af8955-f35c-497d-b88e-d05e5ccb30cc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:26.348372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:26.350533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:26.351101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:26.352465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:26.353653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:26.355654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d5be-c6b0-48a1-9703-dd1ea32846df admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:26.403269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 881/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:26.409070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:26.424121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f4a3e-9a23-4f95-a2c5-588e0e18c887 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:26.440809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:26.442947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:26.443122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:26.443324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] PATH_INFO: `/v3/projects/c6bfefa2e4504084bc5c0beb657ac02a/users/5fd9334cf175450e8da960350edc5e06/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:26.509591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bfefa2e4504084bc5c0beb657ac02a, user_id=5fd9334cf175450e8da960350edc5e06, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:26.512661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:26.553120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 885/1766] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:26 2022] PUT /identity/v3/projects/c6bfefa2e4504084bc5c0beb657ac02a/users/5fd9334cf175450e8da960350edc5e06/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:26.565215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:26.565511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:26.565894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:26.641162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] MFA Rules not processed for user `5fd9334cf175450e8da960350edc5e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:26.642434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:26.878677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 882/1767] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:26.878801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.008375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743b76a-adf5-4109-9f89-69be4829ee53 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.022936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.023424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.026241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.027467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.030247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.032209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.120732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 886/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:26 2022] POST /identity/v3/projects => generated 367 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:27.126740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.157967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b05ec7-da1d-4f31-89de-34190671ee26 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.273291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.274033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.274602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.274878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.275523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.277457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.343465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 883/1769] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:27 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:27.347616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.376154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bab630-206a-4ff8-9a16-a35a2be63d8b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.382505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.383457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.383989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.384413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.385335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.387250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.400252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 887/1770] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:27.406608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.412541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:27.420976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdb9ef-1ddb-4f6b-93eb-d027474906b7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.433918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.435030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.436614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.437151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] PATH_INFO: `/v3/projects/c769076f8ecd40feab8b5e4312b19933/users/2c8e3a47a7fc41bca3f0fa3d62d899a1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.483341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c769076f8ecd40feab8b5e4312b19933, user_id=2c8e3a47a7fc41bca3f0fa3d62d899a1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.486356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.523465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 884/1771] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:27 2022] PUT /identity/v3/projects/c769076f8ecd40feab8b5e4312b19933/users/2c8e3a47a7fc41bca3f0fa3d62d899a1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:27.527321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.545117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9b76-b3a4-4118-ba0a-a9751f14fe96 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.696611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.697389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.697779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.698138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.699264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.703779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15604-4531-40c9-bef6-af08badd70cd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.721617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 888/1772] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:37:27.726230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:27.740686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8f580-b94c-44f1-be11-14286d4ba74d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:27.752608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.753156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.753780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.757316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:27.758140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.758511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.758883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] PATH_INFO: `/v3/projects/c769076f8ecd40feab8b5e4312b19933/users/2c8e3a47a7fc41bca3f0fa3d62d899a1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.766275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:27.773068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c769076f8ecd40feab8b5e4312b19933, user_id=2c8e3a47a7fc41bca3f0fa3d62d899a1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:27.776100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28898ad5-30e7-441f-b262-b52cba29f425 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:27.794652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] MFA Rules not processed for user `930cbe41fc0741cfa5fbd81645181418`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:27.797800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 885/1773] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:27 2022] PUT /identity/v3/projects/c769076f8ecd40feab8b5e4312b19933/users/2c8e3a47a7fc41bca3f0fa3d62d899a1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:27.799876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:27.808842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:27.809278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:27.809549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:27.874954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] MFA Rules not processed for user `2c8e3a47a7fc41bca3f0fa3d62d899a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:27.875915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:27.949039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 889/1774] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:28.047184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 886/1775] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:28.068036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:28.069879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:28.164649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba4c37-9b72-4db0-b627-8d1603b9d572 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:28.169289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:28.170427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:28.170955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:28.171451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:28.184658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:28.187036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8af763-55b6-4aab-8926-188521665c6a service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:28.214953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 890/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:28.268846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:28.271577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:28.286311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28b9e-009a-4c70-9b1f-873068dc6aeb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:28.290938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:28.292191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:28.292578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:28.292729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:28.366201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:28.369041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:28.386509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 887/1777] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:28.995915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:28.996464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:28.996812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:29.146167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:29.147431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:29.318900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 891/1778] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:29.510204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:29.512173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:29.662225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:29.868784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d698-1485-48f8-8f7c-6d25b89ba67d service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:29.902990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:29.903846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:29.903965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:29.909541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:29.936955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:29.943397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31300cdb-4db2-49ec-a4a8-aa54ac5210a4 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:29.967807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 888/1779] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 15:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 15:37:33.440383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:33.496815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:33.541554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a456afc-29b6-4cec-90a1-e01b2ffdb939 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:33.546381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:33.547019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:33.547325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:33.547586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] PATH_INFO: `/v3/users/5fd9334cf175450e8da960350edc5e06` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:33.548710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd9334cf175450e8da960350edc5e06)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:33.550743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:33.604384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd9334cf175450e8da960350edc5e06'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:33.605044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd9334cf175450e8da960350edc5e06'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:33.605654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd9334cf175450e8da960350edc5e06'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:33.612614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd9334cf175450e8da960350edc5e06'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:33.625492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd9334cf175450e8da960350edc5e06'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:33.638089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 892/1780] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:33 2022] DELETE /identity/v3/users/5fd9334cf175450e8da960350edc5e06 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:33.643244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:33.643778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:33.643874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:33.665590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:33.666846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:33.725806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 889/1781] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:33.735655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:33.737048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:33.796964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49763af6-25fa-4cde-9a4b-6fd1491929e8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:33.800608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:33.801221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:33.801506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:33.801835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:33.811381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:33.812809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:33.822445 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 893/1782] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:34.086408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:34.087830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:34.099993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792935a-eae5-4dad-9938-607e004c4226 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:34.104907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:34.105765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:34.106053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:34.106331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] PATH_INFO: `/v3/projects/c6bfefa2e4504084bc5c0beb657ac02a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:34.107971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6bfefa2e4504084bc5c0beb657ac02a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:34.110127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:34.160676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6bfefa2e4504084bc5c0beb657ac02a', 'request_id': 'req-bc0e5f28-b45d-4ba9-9321-f6647056c23b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.166972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6bfefa2e4504084bc5c0beb657ac02a', 'request_id': 'req-bc0e5f28-b45d-4ba9-9321-f6647056c23b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.173364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6bfefa2e4504084bc5c0beb657ac02a', 'request_id': 'req-bc0e5f28-b45d-4ba9-9321-f6647056c23b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.188350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] The token cache is being invalidate because project c6bfefa2e4504084bc5c0beb657ac02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:34.188628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.189263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.190944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 890/1783] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:34 2022] DELETE /identity/v3/projects/c6bfefa2e4504084bc5c0beb657ac02a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:34.194903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:34.204615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade2184-57b4-4407-b4f8-e900e46271fd service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:34.252490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:34.269612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:34.270444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:34.270817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:34.271789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] PATH_INFO: `/v3/users/2c8e3a47a7fc41bca3f0fa3d62d899a1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:34.273618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8e3a47a7fc41bca3f0fa3d62d899a1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:34.275782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:34.327982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8e3a47a7fc41bca3f0fa3d62d899a1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.328634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8e3a47a7fc41bca3f0fa3d62d899a1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.329288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8e3a47a7fc41bca3f0fa3d62d899a1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.336362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8e3a47a7fc41bca3f0fa3d62d899a1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.348791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5586c821-b0a6-4721-89ed-b90f6db74327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8e3a47a7fc41bca3f0fa3d62d899a1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.362283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 894/1784] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:34 2022] DELETE /identity/v3/users/2c8e3a47a7fc41bca3f0fa3d62d899a1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:34.778584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:34.780760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:34.795498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5f28-b45d-4ba9-9321-f6647056c23b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:34.837284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:34.855175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:34.855842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:34.856153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:34.856428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] PATH_INFO: `/v3/projects/c769076f8ecd40feab8b5e4312b19933` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:34.858117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c769076f8ecd40feab8b5e4312b19933)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:34.860074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:34.906445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c769076f8ecd40feab8b5e4312b19933', 'request_id': 'req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.912604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c769076f8ecd40feab8b5e4312b19933', 'request_id': 'req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.918230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c769076f8ecd40feab8b5e4312b19933', 'request_id': 'req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.930158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] The token cache is being invalidate because project c769076f8ecd40feab8b5e4312b19933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:34.930451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.931148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00a5cdb9-7501-48ba-9bdc-8622a4a27ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:34.932672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 891/1785] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:34 2022] DELETE /identity/v3/projects/c769076f8ecd40feab8b5e4312b19933 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:34.938200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:34.938669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:34.938760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:34.959783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:34.960990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:35.040041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 895/1786] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:35.048969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.049456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.050005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.096112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:35.097415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:35.110561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 892/1787] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:35.111307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.122028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de9a7d-0f8b-442a-8964-934839e59699 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.127461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.128067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.128375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.128648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.129306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:35.129724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.131717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.145358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 896/1788] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:37:35.149520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.160377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c0d8e-2df5-4a3b-ae81-8e830e475077 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.165997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.166681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.167052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.167330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.167999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.170203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.227504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 893/1789] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:35 2022] POST /identity/v3/projects => generated 391 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:35.236072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.268414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937b91d-b467-4101-9457-4b06eac06c9c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.329384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:35.351768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.352571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.353005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.353439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.354530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.356929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.396878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 897/1790] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:35 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:35.398167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.411841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d955d-a3a4-47a3-9aa9-0f62a95dcac5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.429613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.430276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.430557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.430833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.431489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.433137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.445830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 894/1791] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:35.449574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.462333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add65759-7b75-4b01-bbf9-eee0fbe4e11a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.469000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.470184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.470539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.470952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.471793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.473771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a28995-c1ae-40fe-b268-46946f76bcc0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.487527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 898/1792] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:35.492661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.509151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f96b26-1cdc-46a4-957f-4e7feb1a4160 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.518313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.522660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.522952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.523304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] PATH_INFO: `/v3/projects/e323b6b492854554a543c579e67f517d/users/a0853573fc284f0d8e1bccd776d807d9/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.543037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e323b6b492854554a543c579e67f517d, user_id=a0853573fc284f0d8e1bccd776d807d9, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.546944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:35.568490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 895/1793] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:35 2022] PUT /identity/v3/projects/e323b6b492854554a543c579e67f517d/users/a0853573fc284f0d8e1bccd776d807d9/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:35.582704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.582992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.585619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.637115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] MFA Rules not processed for user `a0853573fc284f0d8e1bccd776d807d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:35.638404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:35.773444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 899/1794] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:35.778224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:35.940192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0739e4-7ff0-4e79-ad2c-9d7ed117995a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:35.965331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:35.966173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:35.966460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:35.967344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:35.968239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:35.971674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.013428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:36.057540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 896/1795] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:35 2022] POST /identity/v3/projects => generated 391 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:36.062992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.082190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0e685-f46f-44ab-8e47-dcc72868120b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:36.271314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:36.272060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.272517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.272887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.273678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:36.276235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.328043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 900/1796] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:36 2022] POST /identity/v3/users => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:36.329176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.344039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2ec0-eef3-40f4-9b66-a25dc32361ba admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:36.354452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:36.355140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.355444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.356884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.357924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:36.361683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.399220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 897/1797] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:36.405204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.422189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071471f1-317a-43b2-b35a-a28b22807d05 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:36.429435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:36.430784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.431267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.431455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] PATH_INFO: `/v3/projects/2bf4f6d90ebb4d95aa4363209f186207/users/18f1d94a3d9c4aa2a126714370f0484d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.482859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf4f6d90ebb4d95aa4363209f186207, user_id=18f1d94a3d9c4aa2a126714370f0484d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:36.487069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.506359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 901/1798] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:36 2022] PUT /identity/v3/projects/2bf4f6d90ebb4d95aa4363209f186207/users/18f1d94a3d9c4aa2a126714370f0484d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:36.510480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.533461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ab8e-02d0-44d9-a6fe-515467fda59b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:36.627281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:36.627835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.628424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.628424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.629119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:36.631008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b46c2-2845-4bc9-baf8-567452113fab admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.647702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 898/1799] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:37:36.653335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.671191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18aee-761c-4166-8c12-29222b22b336 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:36.677114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:36.677854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.678189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.678460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] PATH_INFO: `/v3/projects/2bf4f6d90ebb4d95aa4363209f186207/users/18f1d94a3d9c4aa2a126714370f0484d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.680631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf4f6d90ebb4d95aa4363209f186207, user_id=18f1d94a3d9c4aa2a126714370f0484d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:36.684129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:36.700638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 902/1800] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:36 2022] PUT /identity/v3/projects/2bf4f6d90ebb4d95aa4363209f186207/users/18f1d94a3d9c4aa2a126714370f0484d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:36.714380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:36.714826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:36.715135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:36.744940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] MFA Rules not processed for user `18f1d94a3d9c4aa2a126714370f0484d`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:36.746817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:36.889628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 899/1801] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:36.900362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:36.904474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:36.947839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:37.018379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13900a92-01ab-4b7c-95a6-983f9dec2b54 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:37.022441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:37.023321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:37.023735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:37.024085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:37.042792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:37.044725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95b2cd-9bf8-4655-847a-bb123d2633c4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:37.078124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 903/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:36 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:37.147467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:37.149019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:37.162853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac465187-9f87-4cb9-aa71-ca97b561753e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:37.166824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:37.167676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:37.167979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:37.168263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:37.277442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:37.280050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:37.305480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 900/1803] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:43.198931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:43.200572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:43.201905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:43.292286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:43.302578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:43.507196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:43.519261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:43.579570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 904/1804] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Sep 12 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:43.690849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c711ff2-931f-4f2c-b6d8-ea0f145a0f3b service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:43.696342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:43.696993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:43.697282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:43.697562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] PATH_INFO: `/v3/users/a0853573fc284f0d8e1bccd776d807d9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:43.698746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0853573fc284f0d8e1bccd776d807d9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:43.700773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:43.758978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:43.760550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:43.781420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580fcbb-8351-4d32-9afc-481c97ef83fb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:43.806469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:43.810040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:43.810929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:43.811217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:43.835263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0853573fc284f0d8e1bccd776d807d9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:43.838017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0853573fc284f0d8e1bccd776d807d9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:43.839830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0853573fc284f0d8e1bccd776d807d9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:43.847064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:43.852056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:43.871938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0853573fc284f0d8e1bccd776d807d9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:43.873883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 905/1805] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:43.875631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:43.878581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:43.901231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad2ab-939a-4a78-a3b4-1d6fb16a3a91 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:43.918377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0853573fc284f0d8e1bccd776d807d9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:43.923163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:43.924699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:43.924699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:43.925143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:43.928731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:43.950691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:43.953868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34211b0-c5e3-4810-8bb0-6e872d0195a3 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:43.974617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 906/1806] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:43.984915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 901/1807] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:43 2022] DELETE /identity/v3/users/a0853573fc284f0d8e1bccd776d807d9 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:43.991884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:43.991884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:43.991884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:44.024695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:44.029805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:44.224666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 907/1808] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:44.246223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:44.247926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:44.381219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1181ae5-71c9-4ed3-8327-cc4890d1019d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:44.386298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:44.386991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:44.387272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:44.387551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:44.406586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:44.408372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf56655-754d-4436-bbe3-701311345ed1 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:44.426670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 902/1809] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:46.624228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:46.627057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:46.656573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67237e-8158-4ff4-959c-91c8ab603c67 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:46.664716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:46.665951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:46.666678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:46.667671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] PATH_INFO: `/v3/projects/e323b6b492854554a543c579e67f517d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:46.669286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e323b6b492854554a543c579e67f517d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:46.672261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:46.734485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:46.751774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e323b6b492854554a543c579e67f517d', 'request_id': 'req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:46.777467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e323b6b492854554a543c579e67f517d', 'request_id': 'req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:46.780515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070638b-b361-44be-bf4d-e66986304737 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:46.781014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070638b-b361-44be-bf4d-e66986304737 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:46.781095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070638b-b361-44be-bf4d-e66986304737 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:46.783284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 903/1810] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:37:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:37:46.790775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e323b6b492854554a543c579e67f517d', 'request_id': 'req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:46.814359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] The token cache is being invalidate because project e323b6b492854554a543c579e67f517d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:46.814954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:46.815086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:46.815243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:46.815686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:46.819477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:46.828960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 908/1811] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:46 2022] DELETE /identity/v3/projects/e323b6b492854554a543c579e67f517d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:46.843470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:46.846776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:46.873617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:46.879359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:46.920198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:46.989725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a0f6e-1c9f-4a42-bd5c-54b9f586e990 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:46.997046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:46.998546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:46.999020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:46.999430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] PATH_INFO: `/v3/users/18f1d94a3d9c4aa2a126714370f0484d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.003712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f1d94a3d9c4aa2a126714370f0484d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:47.003712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:47.007585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 904/1812] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:47.008640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:47.012119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.032265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22145df1-1da0-4410-a8e8-b6f3f6f42bc5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:47.038625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:47.039507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.039997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.040255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] PATH_INFO: `/v3/users/8ad0d0b6886d4d16b2cb2a5165be763b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.041939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad0d0b6886d4d16b2cb2a5165be763b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:47.043988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:47.091211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f1d94a3d9c4aa2a126714370f0484d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.092274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f1d94a3d9c4aa2a126714370f0484d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.093174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f1d94a3d9c4aa2a126714370f0484d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.104814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f1d94a3d9c4aa2a126714370f0484d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.121428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f1d94a3d9c4aa2a126714370f0484d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.132917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad0d0b6886d4d16b2cb2a5165be763b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.133626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad0d0b6886d4d16b2cb2a5165be763b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.134434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad0d0b6886d4d16b2cb2a5165be763b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.146562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad0d0b6886d4d16b2cb2a5165be763b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.147642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 909/1813] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:46 2022] DELETE /identity/v3/users/18f1d94a3d9c4aa2a126714370f0484d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:47.162956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9c0be649-206c-419f-976d-be1e2228120a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad0d0b6886d4d16b2cb2a5165be763b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:47.192549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:47.192549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.193189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 905/1814] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:47 2022] DELETE /identity/v3/users/8ad0d0b6886d4d16b2cb2a5165be763b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:47.197994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.198450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.198838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.229919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:47.232566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.261722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:47.336249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf1b2-844d-4c92-803c-24f44294c429 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:47.392950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 906/1815] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:47.439952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:47.440715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:47.441650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.441945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.442285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.442827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.445871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.462665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:47.464749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b1b09-d29f-444f-96cf-738d3c25b22f service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:47.488644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:47.489995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 910/1816] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:47.535128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d766-75a2-48e7-87da-e27280806d2c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:47.539899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:47.540863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.541058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.541342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.554762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:47.564082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb0311-16f5-4527-980e-391114f68c30 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:47.590950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 907/1817] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:47.795369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.795950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.795950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.797933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 911/1818] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:37:47.867865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:47.868492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:47.868722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:47.893265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:47.894717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:47.906899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 908/1819] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:48.302973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:48.306064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:48.332586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54e75-0f4f-48a6-a1b0-cce33c19797f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:48.339046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:48.339752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:48.340096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:48.340468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:48.358333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:48.369352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:48.394151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 912/1820] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:37:49.042277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.049262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.088752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17e51c-fea1-492b-a09a-6d673fb11b66 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.091772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.094945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.098443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.099648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.100648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.101188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] PATH_INFO: `/v3/projects/3b6b444e93e44ca382e3116061c33a81` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.103334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6b444e93e44ca382e3116061c33a81)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.108381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.125523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec4e60-5e41-48c6-aaed-9557dcbc8538 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.138248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.139035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.139387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.139737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] PATH_INFO: `/v3/projects/2bf4f6d90ebb4d95aa4363209f186207` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.141430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf4f6d90ebb4d95aa4363209f186207)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.143550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.204113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6b444e93e44ca382e3116061c33a81', 'request_id': 'req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.211252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6b444e93e44ca382e3116061c33a81', 'request_id': 'req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.218572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6b444e93e44ca382e3116061c33a81', 'request_id': 'req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.226100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf4f6d90ebb4d95aa4363209f186207', 'request_id': 'req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.232987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] The token cache is being invalidate because project 3b6b444e93e44ca382e3116061c33a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:49.233482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.234170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.235572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf4f6d90ebb4d95aa4363209f186207', 'request_id': 'req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.235822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 909/1821] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:49 2022] DELETE /identity/v3/projects/3b6b444e93e44ca382e3116061c33a81 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:49.242790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.243316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf4f6d90ebb4d95aa4363209f186207', 'request_id': 'req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.244290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.256213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] The token cache is being invalidate because project 2bf4f6d90ebb4d95aa4363209f186207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:49.256676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.257436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e2b18604-c2d2-47f6-a6c0-4febcc2f2817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.259306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 913/1822] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:49 2022] DELETE /identity/v3/projects/2bf4f6d90ebb4d95aa4363209f186207 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:49.263991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf87c0-653b-49d9-9d92-45af98600bd6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.264385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf87c0-653b-49d9-9d92-45af98600bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.265213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf87c0-653b-49d9-9d92-45af98600bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.288468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-74cf87c0-653b-49d9-9d92-45af98600bd6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:49.290544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74cf87c0-653b-49d9-9d92-45af98600bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.330684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9505-bc5c-4acb-b1df-d8046eb5e9db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.338409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:49.375907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 914/1823] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:49.385298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.385595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.385896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.413382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:49.414653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.417468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:49.429551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.429993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 915/1824] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:49.437636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.437636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.437636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] PATH_INFO: `/v3/users/77422b32b0ae4e5a97351d2c4afd56f1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.437636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77422b32b0ae4e5a97351d2c4afd56f1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.438892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.439017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.448654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6aca4b-a3b7-43d4-a5fd-dcf5b45811d6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.454197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.455057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.455790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.456227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.456890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:49.457521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.459202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.473412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 916/1825] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:37:49.477417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.490659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c8742-db53-48ad-92ec-739337b95c42 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.496035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.496845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.497297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.497785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.498587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.500671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.529676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77422b32b0ae4e5a97351d2c4afd56f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.530454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77422b32b0ae4e5a97351d2c4afd56f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.531246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77422b32b0ae4e5a97351d2c4afd56f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.542239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77422b32b0ae4e5a97351d2c4afd56f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.561855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 917/1826] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:49 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:49.563603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.565662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77422b32b0ae4e5a97351d2c4afd56f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:49.566985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.581338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 910/1827] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:49 2022] DELETE /identity/v3/users/77422b32b0ae4e5a97351d2c4afd56f1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:49.643906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539ed-6673-4e93-b7cc-5d6c9c72760d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.711125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.711816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.712092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.712367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.713056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.715281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.737908 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 918/1828] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:49 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:49.742086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.752917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8f3a2-3b37-4a54-85ad-131296f52938 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.765399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.766076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.766376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.766644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.767445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.769288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.786042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 911/1829] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:49.792076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.803763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd295-b8b5-4f80-87d5-c6e49d6405ff admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.809300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.810012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.810387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.810929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.811644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.813293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba8ec9-61bd-4b3b-8a20-043969ebe48f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.835424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 919/1830] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:49.838415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.849726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b89f5a-c2fc-4bfb-b737-2a7b8d066c76 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:49.854410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:49.855147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.855518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.855861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] PATH_INFO: `/v3/projects/b5cfa94d09c14fc899f75a33151ef0ec/users/3d9a08aed6574b7995438f21ff276510/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.874318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5cfa94d09c14fc899f75a33151ef0ec, user_id=3d9a08aed6574b7995438f21ff276510, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:49.874909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:49.891341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 912/1831] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:49 2022] PUT /identity/v3/projects/b5cfa94d09c14fc899f75a33151ef0ec/users/3d9a08aed6574b7995438f21ff276510/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:49.902200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:49.902501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:49.902799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:49.929218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:49.931415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.940818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] MFA Rules not processed for user `3d9a08aed6574b7995438f21ff276510`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:49.943344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:49.954434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:50.011945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb015174-536e-472e-b7e1-2c12f0c30253 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.017576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.018272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.018561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.019189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] PATH_INFO: `/v3/projects/efc6fd4346eb430488a7a5690bfd62d2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.020019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc6fd4346eb430488a7a5690bfd62d2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.022083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.028281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 920/1832] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:50.036402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.038040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:50.072776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:50.080818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc6fd4346eb430488a7a5690bfd62d2', 'request_id': 'req-325420fb-290f-4b61-842d-b25b479675c9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:50.096888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc6fd4346eb430488a7a5690bfd62d2', 'request_id': 'req-325420fb-290f-4b61-842d-b25b479675c9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:50.104218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc6fd4346eb430488a7a5690bfd62d2', 'request_id': 'req-325420fb-290f-4b61-842d-b25b479675c9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:50.118557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b95565-a937-4978-8762-c8da78ad58fc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.127868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] The token cache is being invalidate because project efc6fd4346eb430488a7a5690bfd62d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:50.128302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:50.129109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-325420fb-290f-4b61-842d-b25b479675c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:50.131380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 913/1833] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:49 2022] DELETE /identity/v3/projects/efc6fd4346eb430488a7a5690bfd62d2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:50.136276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ded3-9ebb-4459-94dd-77b64c17a531 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.136877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ded3-9ebb-4459-94dd-77b64c17a531 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.137274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ded3-9ebb-4459-94dd-77b64c17a531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.162076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4b28ded3-9ebb-4459-94dd-77b64c17a531 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:50.163485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b28ded3-9ebb-4459-94dd-77b64c17a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:50.207227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.207992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.208274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.208601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.210739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:50.233187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 914/1834] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:50.243387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.243783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.244068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.269290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:50.270583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:50.281847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 915/1835] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:50.284352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.287910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.289864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.296420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5332e41-9880-4405-a324-69853fd8cbb1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.301903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.302523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.303987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.304271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.304934 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:50.305246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.305973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 921/1836] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:50.306969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.321546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 916/1837] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:37:50.327177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.339763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c7c89-4046-4ff2-9aa3-3be2a7f3c278 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.345100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.346021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.346313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.346939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.347334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.349292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.384034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:50.397047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 922/1838] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:50.399123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.412736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafe2f0-3cc6-4a72-9ecb-e8ef93998b7f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.480782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.481633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.482061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.482588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.483343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.485887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.509206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 917/1839] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:50.514397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.527405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89d16-5b44-414c-b332-a53bebc89e9d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.533975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.534989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.535475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.536075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.536946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.540648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.555566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 923/1840] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:50.570265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.579898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9df97d-283e-4d82-83d5-28468e42d7ed admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.585499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.586909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.587268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.587618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.588416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.590140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5d620-ba53-4292-96f8-6c22069be52a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.603676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 918/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:50.611108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.627867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712620c-7bc6-4b0d-9eee-e45769ede4f2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.636666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.636666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.636666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.637001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] PATH_INFO: `/v3/projects/c4f6114a3d864ec98bd38998aa596d90/users/89fb4c31bcec4371a162cf779b214014/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.657334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f6114a3d864ec98bd38998aa596d90, user_id=89fb4c31bcec4371a162cf779b214014, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.661040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.686584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 924/1842] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:50 2022] PUT /identity/v3/projects/c4f6114a3d864ec98bd38998aa596d90/users/89fb4c31bcec4371a162cf779b214014/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:50.698258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.698994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.698994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.733121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] MFA Rules not processed for user `89fb4c31bcec4371a162cf779b214014`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:50.734496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:50.753221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:50.803113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 919/1843] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:50.807570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.870083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d945c28-e694-4417-8535-7a73b29e97d9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.875190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.875799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.876083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.876359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.876998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.878941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:50.905835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 925/1844] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:50.909329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:50.919841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fa522-f1f6-457c-ace9-d2e6b05dba85 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:50.975503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:50.976108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:50.976397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:50.976668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:50.977291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:50.979235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.000154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 920/1845] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:50 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:51.004162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.015048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4510-8cb9-4756-a031-6707cb5cc1ab admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.019630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.020224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.020511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.020788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.021429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.023091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.034549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 926/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:51.038604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.048780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2201f6-097a-481e-b836-e00674fd4b5b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.053334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.054031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.054314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.054595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] PATH_INFO: `/v3/projects/a49769dd51d447c99640cb86466b4834/users/1c8da7f2b24340659bb790f9cbf17e49/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.070152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49769dd51d447c99640cb86466b4834, user_id=1c8da7f2b24340659bb790f9cbf17e49, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.073115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.085694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 921/1847] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:51 2022] PUT /identity/v3/projects/a49769dd51d447c99640cb86466b4834/users/1c8da7f2b24340659bb790f9cbf17e49/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:51.089617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.099444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9600861-9a2d-48db-b2cf-f1996ba64ab7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.160162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.160803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.160977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.161254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.161933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.163709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606dce3-b665-4bf0-95be-b80dbff104ae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.178190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 927/1848] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:51.178763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.189800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfe3e6-7444-448c-a27a-6d043727c56d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.194396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.195056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.195337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.195884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] PATH_INFO: `/v3/projects/a49769dd51d447c99640cb86466b4834/users/1c8da7f2b24340659bb790f9cbf17e49/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.197795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49769dd51d447c99640cb86466b4834, user_id=1c8da7f2b24340659bb790f9cbf17e49, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.200455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.210850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 922/1849] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:51 2022] PUT /identity/v3/projects/a49769dd51d447c99640cb86466b4834/users/1c8da7f2b24340659bb790f9cbf17e49/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:51.221974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.222265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.222545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.251870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] MFA Rules not processed for user `1c8da7f2b24340659bb790f9cbf17e49`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:51.253146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:51.273621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:51.324426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 928/1850] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:51.337381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.339061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:51.398864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35bf0-01d9-4181-8bf3-bfd436d18fd4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.402467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.403076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.403359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.403632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.413025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.414431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.425070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 923/1851] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:51.543796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:51.554246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fedaf8-01d6-412a-aae9-7a58727bf412 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:51.557946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:51.558646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:51.558997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:51.559336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:51.630110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:51.631568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d18ea-84ce-4e3e-a5e5-33fd8b686658 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:51.640572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 929/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:56.710285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:56.710285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:56.740418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:56.844262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4d040-518f-4e70-8987-f358c543c3bf service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:56.851197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:56.851903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:56.852229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:56.852826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] PATH_INFO: `/v3/users/3d9a08aed6574b7995438f21ff276510` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:56.854740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9a08aed6574b7995438f21ff276510)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:56.856749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:56.946416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9a08aed6574b7995438f21ff276510'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:56.947332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9a08aed6574b7995438f21ff276510'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:56.948156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9a08aed6574b7995438f21ff276510'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:56.958971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9a08aed6574b7995438f21ff276510'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:56.976544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9a08aed6574b7995438f21ff276510'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.028244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 924/1853] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:56 2022] DELETE /identity/v3/users/3d9a08aed6574b7995438f21ff276510 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:57.029358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.029860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.030567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.055580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:57.057312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.121987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 930/1854] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:57.132236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.133692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.195350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ba2e-c560-4e53-be44-1d1fdbc6966f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:57.199845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:57.200004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.200154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.200432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.211783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:57.213239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad858ee8-1a96-430b-8add-8affab34ab26 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:57.223659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 925/1855] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:57.497036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.498484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.514143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620cb85-7ef3-4a3f-be11-d9703e9b8910 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:57.519793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:57.520440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.520812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.520991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] PATH_INFO: `/v3/projects/b5cfa94d09c14fc899f75a33151ef0ec` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.522293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5cfa94d09c14fc899f75a33151ef0ec)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:57.524086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:57.571933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5cfa94d09c14fc899f75a33151ef0ec', 'request_id': 'req-308e422b-4f41-4bb1-82c0-05aff705508d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.574487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:57.581560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5cfa94d09c14fc899f75a33151ef0ec', 'request_id': 'req-308e422b-4f41-4bb1-82c0-05aff705508d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.588968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5cfa94d09c14fc899f75a33151ef0ec', 'request_id': 'req-308e422b-4f41-4bb1-82c0-05aff705508d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.599495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] The token cache is being invalidate because project b5cfa94d09c14fc899f75a33151ef0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:57.600126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.600756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-308e422b-4f41-4bb1-82c0-05aff705508d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.602287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 931/1856] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:57 2022] DELETE /identity/v3/projects/b5cfa94d09c14fc899f75a33151ef0ec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:57.607113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.607410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.607695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.628582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:57.629885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.663172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:57.696799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 926/1857] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:57.705037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.705395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.705793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.743193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:57.744784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.762338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 932/1858] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:57.766725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.781901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.783951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a2bb8-1098-486e-82e7-ef33d53a2765 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:57.784503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.797695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:57.797959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.798109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.798515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.799842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:37:57.799842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:57.803268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:57.812176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36d1c2-d7c3-4d39-bb4f-52b9b9ea26d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:57.821436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:57.823304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.823604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.823885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] PATH_INFO: `/v3/users/89fb4c31bcec4371a162cf779b214014` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.832846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fb4c31bcec4371a162cf779b214014)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:57.834164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 927/1859] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:37:57.835083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.837453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:57.863361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7877c-8685-4947-a66c-79584eab3523 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:57.870624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:57.871258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:57.871547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:57.871826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:57.872794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:57.879570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:57.947417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 928/1860] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:57 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:57.953790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:57.957054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fb4c31bcec4371a162cf779b214014'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.959482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fb4c31bcec4371a162cf779b214014'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.961277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:57.962307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fb4c31bcec4371a162cf779b214014'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.973305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fb4c31bcec4371a162cf779b214014'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:57.992324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a38e1a0c-af77-4df3-ae2c-5b4a2086b9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fb4c31bcec4371a162cf779b214014'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:58.011890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 933/1861] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:57 2022] DELETE /identity/v3/users/89fb4c31bcec4371a162cf779b214014 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:58.016727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.017428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.017550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.086025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:58.096302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b092df-2fc7-42cd-8c82-677fc108fd9d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.098811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:58.196029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 934/1862] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:58.197024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.198467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:58.240913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.241867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.242318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.242751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.245371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.250130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.261557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:58.270747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b067-fec9-44c5-a4d2-2736b265596c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.282774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.282774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.282774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.282774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.303408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.305123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.306060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 929/1863] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:57 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:58.312210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.325853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4005f0c-a887-4091-8bfd-7c8612ade2fa admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.329853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 935/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:58.337514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.338637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.339268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.339933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.341838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.344326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c28af2-ed36-4147-8370-11e0a2a4f636 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.369194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 930/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:37:58.376360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.391644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89772d28-f657-4a50-bb43-baf31d45b31c service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.397235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.398256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.398744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.399151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.400014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.402185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea23276-ae12-475e-94b3-bf895e114f4c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.412029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c1107-54e3-42d3-9599-b701831ee687 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.412542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c1107-54e3-42d3-9599-b701831ee687 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.413194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c1107-54e3-42d3-9599-b701831ee687 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.415645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 931/1866] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:37:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:37:58.418438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 936/1867] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:58.423158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c1107-54e3-42d3-9599-b701831ee687 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.443681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c1107-54e3-42d3-9599-b701831ee687 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.450611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.451252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.451804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.470692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.474112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.474854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.475886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] PATH_INFO: `/v3/projects/7e67b54b9de24a65b9d350d9a99bd1f9/users/f8d79ac5301c4581b6b7faac4bf78b72/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.501982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e67b54b9de24a65b9d350d9a99bd1f9, user_id=f8d79ac5301c4581b6b7faac4bf78b72, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.505467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27494f56-2a04-490d-8426-a244d0e8fe63 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.508678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:58.512247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:58.578305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 932/1868] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:58 2022] PUT /identity/v3/projects/7e67b54b9de24a65b9d350d9a99bd1f9/users/f8d79ac5301c4581b6b7faac4bf78b72/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:58.594791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.595433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.595935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.607389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:58.663368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] MFA Rules not processed for user `f8d79ac5301c4581b6b7faac4bf78b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:58.667237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:58.726797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 937/1869] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:58.803672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.803849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 933/1870] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:37:58.805500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:58.807140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:58.832218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d367110-55f4-4fb2-bb5b-e8949aee8c2e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.923622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecf46d-ef4c-4876-8c7b-8b7ec5e8c043 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:58.942208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.945052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.945354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.946086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.947296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.949340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:58.950603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:58.951331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:58.951778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] PATH_INFO: `/v3/projects/c4f6114a3d864ec98bd38998aa596d90` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:58.953258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:58.955491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f6114a3d864ec98bd38998aa596d90)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:58.957937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.011186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 934/1871] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:58 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:59.012179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.013843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.031821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f6114a3d864ec98bd38998aa596d90', 'request_id': 'req-73a5a46c-7a71-4ab9-aebc-e783f068550d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.043833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f6114a3d864ec98bd38998aa596d90', 'request_id': 'req-73a5a46c-7a71-4ab9-aebc-e783f068550d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.051133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f6114a3d864ec98bd38998aa596d90', 'request_id': 'req-73a5a46c-7a71-4ab9-aebc-e783f068550d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.063204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:59.069435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] The token cache is being invalidate because project c4f6114a3d864ec98bd38998aa596d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:59.070264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.071952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.073796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 938/1872] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:58 2022] DELETE /identity/v3/projects/c4f6114a3d864ec98bd38998aa596d90 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:59.078695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.080664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.104347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9ae0-d7b1-47fc-933c-ee2b1ee256e4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.118064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:59.152034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a46c-7a71-4ab9-aebc-e783f068550d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.156800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.157410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.157692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.157993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.158622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.160543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.177004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.177628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.177955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.178237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.179985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.181060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 939/1873] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:59 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:37:59.181962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.185140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.192669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.194335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.197736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4d39-7013-47a0-ab92-9ccc225253f4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.202962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.203575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.203860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.204137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] PATH_INFO: `/v3/users/1c8da7f2b24340659bb790f9cbf17e49` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.205261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8da7f2b24340659bb790f9cbf17e49)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.205781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 935/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:37:59.206438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.207213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.216303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e897a-4fb0-4930-b031-8aca0ef944fd service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.221255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.221893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.222261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.222446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.223093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.224685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.235110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 936/1875] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:59.240204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.253202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12cf5d-59e6-4026-b15d-c94f6a4060ed admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.261643 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8da7f2b24340659bb790f9cbf17e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.262420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8da7f2b24340659bb790f9cbf17e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.263048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8da7f2b24340659bb790f9cbf17e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.267262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.267980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.268171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.268451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.269104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.270778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.270903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8da7f2b24340659bb790f9cbf17e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.281066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 937/1876] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:37:59.283990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6be049c1-52ac-4223-9117-b9350aa67897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8da7f2b24340659bb790f9cbf17e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.284955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.286723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.297693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 940/1877] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:37:59 2022] DELETE /identity/v3/users/1c8da7f2b24340659bb790f9cbf17e49 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:59.300399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a3b7c-1001-48b7-9ce2-ef4293277d5a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.361149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.361843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.362213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.362549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] PATH_INFO: `/v3/projects/78b2653c21074cd7b547a5099ec086f5/users/cab16c35892d410e8556becc422364ad/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.379973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b2653c21074cd7b547a5099ec086f5, user_id=cab16c35892d410e8556becc422364ad, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.382747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.397308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 938/1878] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:37:59 2022] PUT /identity/v3/projects/78b2653c21074cd7b547a5099ec086f5/users/cab16c35892d410e8556becc422364ad/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:37:59.409109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.409501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.409721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.443437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] MFA Rules not processed for user `cab16c35892d410e8556becc422364ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:59.445085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.506573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:59.529514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 941/1879] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:37:59.534095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.632786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d372e-2ac4-4695-b58d-003846099a23 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.638562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.639336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.639723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.640432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.641676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.644007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.651073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:37:59.684190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.686315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:37:59.708773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c9e4a-bde2-4e8a-8de9-ed9f85101f33 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.720389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 939/1880] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:37:59.726986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:37:59.727131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:37:59.727963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.728426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.728759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] PATH_INFO: `/v3/projects/a49769dd51d447c99640cb86466b4834` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.730026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49769dd51d447c99640cb86466b4834)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:37:59.732407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:37:59.818731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49769dd51d447c99640cb86466b4834', 'request_id': 'req-062a686a-1548-4504-8fc2-c60e586f8974'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.841469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49769dd51d447c99640cb86466b4834', 'request_id': 'req-062a686a-1548-4504-8fc2-c60e586f8974'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.852280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49769dd51d447c99640cb86466b4834', 'request_id': 'req-062a686a-1548-4504-8fc2-c60e586f8974'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.873252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64a1cb-dc2f-4db0-ab76-8a6c98c86d7a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:37:59.875466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] The token cache is being invalidate because project a49769dd51d447c99640cb86466b4834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:37:59.875768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.877284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-062a686a-1548-4504-8fc2-c60e586f8974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:37:59.883190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 942/1881] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:37:59 2022] DELETE /identity/v3/projects/a49769dd51d447c99640cb86466b4834 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:37:59.886775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f30c7-22c1-47ab-8ce4-6c917ce25690 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:37:59.887573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f30c7-22c1-47ab-8ce4-6c917ce25690 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:37:59.887573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f30c7-22c1-47ab-8ce4-6c917ce25690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:37:59.930392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-250f30c7-22c1-47ab-8ce4-6c917ce25690 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:37:59.931677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-250f30c7-22c1-47ab-8ce4-6c917ce25690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:00.017476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.018287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.018583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.018853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.019505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.024480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.073311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 940/1882] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:37:59 2022] POST /identity/v3/users => generated 298 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:00.078052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.079804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 943/1883] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:00.087284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:00.091622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.093513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.094282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.100071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a03950-386f-4308-a7df-8e8b40ea1d67 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.104957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.105689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.106096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.106454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.107183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.109149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.124791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:00.125588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 941/1884] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:00.125789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:00.126893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.137952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216046bc-ac15-4e98-a68a-78066b96e8c3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.142011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 944/1885] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:00.146193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.150336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.151462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.151962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.152455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] PATH_INFO: `/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864/users/107ea84531704f9d83fc307d1b209fa0/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.159512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ca86-869b-4ed6-82c6-9f74549754d8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.170935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.171581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.171905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.172316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.173936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:00.174294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.177317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.177745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0d9c2290ac48a19b5737faa8ec7864, user_id=107ea84531704f9d83fc307d1b209fa0, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.181723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.201481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 945/1886] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:00.205695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 942/1887] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:00 2022] PUT /identity/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864/users/107ea84531704f9d83fc307d1b209fa0/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:00.210810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.212170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.228558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ed0af-f226-4397-b74f-de9d2c76fc28 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.336604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:00.357378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.358937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:00.359856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.361053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.361053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.362064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.365827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.398914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8b857-91f5-4c80-865d-e8b59aaab97f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.404337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.405916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.406191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.407463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.408753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.410690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.427760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 943/1888] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:00.429646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.431944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 946/1889] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:00 2022] POST /identity/v3/projects => generated 409 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:00.437937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.448813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca105067-17bc-48c4-a916-11dee83a5d4f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.549801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.551346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.551771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.552175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] PATH_INFO: `/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864/users/107ea84531704f9d83fc307d1b209fa0/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.555088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0d9c2290ac48a19b5737faa8ec7864, user_id=107ea84531704f9d83fc307d1b209fa0, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.558837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc7af3-1823-4476-8f76-318355e65157 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.581590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534053c9-49ca-4733-b1f8-89a19db47705 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.584168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 944/1890] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:00 2022] PUT /identity/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864/users/107ea84531704f9d83fc307d1b209fa0/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:00.596422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.596728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.597003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.638133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] MFA Rules not processed for user `107ea84531704f9d83fc307d1b209fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:00.640219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:00.731774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.732412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.732944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.733805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.734583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.736944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.772467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 947/1891] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:00 2022] POST /identity/v3/users => generated 305 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:00.774498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.782246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 945/1892] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:00.788435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4224d-91c7-4332-bea4-b14786158cfe admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.806614 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.809834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:00.813892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.814564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.814850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.815125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.815775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.817387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.831319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 948/1893] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:00.836222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.850366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c0b49-8e68-47a2-9f28-67591dbc0fd5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.856426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.857334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.859614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.859968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.861067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.862780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.875555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 949/1894] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:00.880920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:00.895817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6a5b5-add3-4fff-997d-531728f6f4df admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:00.901789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:00.902459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.902739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.903012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] PATH_INFO: `/v3/projects/c32986f912384c119f312d44919a689b/users/824eee1332b24e42a6575c1f36363fb3/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.920421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32986f912384c119f312d44919a689b, user_id=824eee1332b24e42a6575c1f36363fb3, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:00.924064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad1918-5acf-410f-a586-c976cd323a31 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:00.943804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 950/1895] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:00 2022] PUT /identity/v3/projects/c32986f912384c119f312d44919a689b/users/824eee1332b24e42a6575c1f36363fb3/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:00.956296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:00.956679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:00.957218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:00.967080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214afe2-b82d-494d-975c-39ff5ca18814 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:01.001650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] MFA Rules not processed for user `824eee1332b24e42a6575c1f36363fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:01.002948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:01.017732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:01.059760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:01.150489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:01.151457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:01.151760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:01.151931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:01.153760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:01.198492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 951/1896] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:01.199756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:01.287486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:01.288582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:01.313592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 946/1897] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:01.358943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e643f1-b5a2-476a-ae54-fe3e58de7540 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:01.384777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:01.386406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:01.386968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:01.388289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:01.389056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:01.395086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:01.465982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:01.466721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 952/1898] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:01 2022] POST /identity/v3/projects => generated 407 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:01.484408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e11f6-f968-49d0-92ec-99ae245a895b service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:01.610008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:01.610736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:01.611189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:01.612019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:01.612307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:01.614504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:01.650014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 947/1899] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:01 2022] POST /identity/v3/users => generated 310 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:01.653232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:01.668614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235c8bb-0f02-4b44-b65b-0ef3bd2bd6c4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:01.675417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:01.676247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:01.684582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:01.689668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:01.690537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:01.693618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:01.716282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 953/1900] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:01.735596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:01.749985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f46e7b-3b57-42b5-9841-6e4a1ce6f0ec admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:01.773624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:01.774353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:01.774740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:01.774903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] PATH_INFO: `/v3/projects/c99b430909f64f229eb8f603afb52530/users/21e68b72cd3b4ff4a9eed651303b1efa/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:01.825169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99b430909f64f229eb8f603afb52530, user_id=21e68b72cd3b4ff4a9eed651303b1efa, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:01.828850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:01.860583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 948/1901] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:01 2022] PUT /identity/v3/projects/c99b430909f64f229eb8f603afb52530/users/21e68b72cd3b4ff4a9eed651303b1efa/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:01.864691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:01.877684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86941a2a-9b37-4b7d-8fa6-f10c5a0c9aba admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:02.027056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:02.027741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:02.028047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:02.029559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:02.029559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:02.031475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08be7b-6475-4877-8a4b-f232fb7901a1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:02.047539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 954/1902] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:02.051668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:02.073688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7170-a1d8-49f2-b803-22346417d0df admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:02.079084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:02.080161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:02.080472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:02.080730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] PATH_INFO: `/v3/projects/c99b430909f64f229eb8f603afb52530/users/21e68b72cd3b4ff4a9eed651303b1efa/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:02.083068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99b430909f64f229eb8f603afb52530, user_id=21e68b72cd3b4ff4a9eed651303b1efa, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:02.085841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:02.101144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 949/1903] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:02 2022] PUT /identity/v3/projects/c99b430909f64f229eb8f603afb52530/users/21e68b72cd3b4ff4a9eed651303b1efa/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:02.112389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:02.112687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:02.112968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:02.146962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] MFA Rules not processed for user `21e68b72cd3b4ff4a9eed651303b1efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:02.148285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:02.172874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:02.259480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 955/1904] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:02.283542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:02.336669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:02.399698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9abdd-da6e-4ca4-8276-63d788941115 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:02.409595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:02.411621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:02.412018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:02.412386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:02.424953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:02.426694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:02.437721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 950/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:02.658177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:02.676186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4829b8-5da3-4358-a42d-88dccb065523 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:02.681304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:02.682431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:02.682869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:02.683447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:02.772898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:02.774859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:02.786253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 956/1906] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:03.271006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:03.282730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcd53f-6e2d-483c-a665-b799306c6c4d service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:03.287420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:03.288286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:03.288785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:03.289349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:03.291441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:03.444927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:03.446787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:03.474020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 951/1907] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:04.076707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:04.086149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c3d61-d892-49b4-8ff6-4846601d9eba service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:04.090084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:04.090748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:04.091107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:04.091459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:04.153594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:04.155207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f4ffe-629e-4753-ad96-f47f30a570b4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:04.165484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 957/1908] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:11.900791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:11.902939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.023003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af141-4a0c-45b6-8d08-e4bc4a015906 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:12.027818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:12.028624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.028906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.029302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] PATH_INFO: `/v3/users/f1cc2e1b76cf423097ae0e8382001046` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.030488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1cc2e1b76cf423097ae0e8382001046)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:12.033081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:12.100693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1cc2e1b76cf423097ae0e8382001046'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.101733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1cc2e1b76cf423097ae0e8382001046'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.102629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1cc2e1b76cf423097ae0e8382001046'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.111621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1cc2e1b76cf423097ae0e8382001046'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.125182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:12.130168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1cc2e1b76cf423097ae0e8382001046'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.149058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 952/1909] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:11 2022] DELETE /identity/v3/users/f1cc2e1b76cf423097ae0e8382001046 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:12.154477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.154985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.154985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.175107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:12.176338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.253556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:12.273154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 958/1910] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:12.283744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:12.285177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.394606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157482b6-acb9-4cd1-95f1-60b602f05283 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:12.400541 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:12.401267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.401592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.401987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.419987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:12.421538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:12.435327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 953/1911] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:12.781463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:12.783762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.796357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d812939-4767-453f-a5a8-b256a5bd2bc8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:12.802765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:12.803560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.803852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.804138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] PATH_INFO: `/v3/users/824eee1332b24e42a6575c1f36363fb3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.805254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=824eee1332b24e42a6575c1f36363fb3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:12.807891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:12.835340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:12.836858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.855600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737416aa-ddf5-45d7-a054-ce0541af8d7c service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:12.860955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:12.861979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.862760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.863338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] PATH_INFO: `/v3/projects/c18510d59b5e42a08cbae34e622a9087` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.864617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18510d59b5e42a08cbae34e622a9087)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:12.866586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:12.873435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824eee1332b24e42a6575c1f36363fb3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.874815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824eee1332b24e42a6575c1f36363fb3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.875752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824eee1332b24e42a6575c1f36363fb3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.885142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824eee1332b24e42a6575c1f36363fb3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.899821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-aab71e0f-2000-47c9-be7c-d9e5fb3be9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824eee1332b24e42a6575c1f36363fb3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.913270 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 959/1912] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:12 2022] DELETE /identity/v3/users/824eee1332b24e42a6575c1f36363fb3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:12.917937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:12.918424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:12.918705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:12.922932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18510d59b5e42a08cbae34e622a9087', 'request_id': 'req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.929270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18510d59b5e42a08cbae34e622a9087', 'request_id': 'req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.935415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18510d59b5e42a08cbae34e622a9087', 'request_id': 'req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.938564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:12.939777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:12.948813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] The token cache is being invalidate because project c18510d59b5e42a08cbae34e622a9087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:12.949109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.949756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:12.951256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 954/1913] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:12 2022] DELETE /identity/v3/projects/c18510d59b5e42a08cbae34e622a9087 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:12.955133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:12.956493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:13.007875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 960/1914] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:13.017434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:13.018916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:13.032884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a8d79-18b8-45dd-a7b5-e4e8139bd6f6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:13.037477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:13.038141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:13.038427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:13.038705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] PATH_INFO: `/v3/users/2c4943a1eed14ae3a51896d48b843596` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:13.039832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4943a1eed14ae3a51896d48b843596)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:13.041986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:13.056219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:13.082134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f4113-7ede-487e-a9ae-c89c6db3bcbe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:13.086732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:13.087492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:13.087778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:13.088050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:13.100194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:13.102165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:13.124207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 961/1915] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:13.131437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4943a1eed14ae3a51896d48b843596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.132230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4943a1eed14ae3a51896d48b843596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.133249 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4943a1eed14ae3a51896d48b843596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.141154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4943a1eed14ae3a51896d48b843596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.161856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4943a1eed14ae3a51896d48b843596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.185056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 955/1916] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:12 2022] DELETE /identity/v3/users/2c4943a1eed14ae3a51896d48b843596 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:13.626440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:13.627927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:13.639419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:13.644121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a5eed-65dc-450d-bba6-2b887011af96 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:13.709595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:13.711347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:13.764669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:13.766753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:13.767088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:13.767835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] PATH_INFO: `/v3/users/f8d79ac5301c4581b6b7faac4bf78b72` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:13.771243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d79ac5301c4581b6b7faac4bf78b72)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:13.776094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:13.881746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230153e0-d0c2-4e51-a00c-b1b329275126 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:13.881746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:13.881746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:13.881746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:13.882150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] PATH_INFO: `/v3/projects/c32986f912384c119f312d44919a689b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:13.882150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32986f912384c119f312d44919a689b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:13.882150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:13.889332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d79ac5301c4581b6b7faac4bf78b72'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.890413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d79ac5301c4581b6b7faac4bf78b72'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.891440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d79ac5301c4581b6b7faac4bf78b72'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.901380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d79ac5301c4581b6b7faac4bf78b72'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.917896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d79ac5301c4581b6b7faac4bf78b72'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:13.946394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 962/1917] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:13 2022] DELETE /identity/v3/users/f8d79ac5301c4581b6b7faac4bf78b72 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:13.949505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:13.949505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:13.994080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32986f912384c119f312d44919a689b', 'request_id': 'req-50fbd020-d6a5-42cf-9c14-2838ed120d82'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.003763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32986f912384c119f312d44919a689b', 'request_id': 'req-50fbd020-d6a5-42cf-9c14-2838ed120d82'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.009627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32986f912384c119f312d44919a689b', 'request_id': 'req-50fbd020-d6a5-42cf-9c14-2838ed120d82'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.026617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] The token cache is being invalidate because project c32986f912384c119f312d44919a689b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:14.027042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.027709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-50fbd020-d6a5-42cf-9c14-2838ed120d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.033096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 956/1918] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:13 2022] DELETE /identity/v3/projects/c32986f912384c119f312d44919a689b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:14.034461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.034749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.035072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.058397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:14.060396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.082005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc787cce-0fff-4f22-be5e-ed4e1b37fb18 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.091800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:14.141487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:14.211152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.211827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 957/1919] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:14.211960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.212118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.212377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] PATH_INFO: `/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.214004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3246ceeb7d64de8b095f5d7c305b7c4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.215061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.216050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.220233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.247960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd4d9-4383-4813-9006-78fdab02e47a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.260523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.261264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.261630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.262025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] PATH_INFO: `/v3/users/21e68b72cd3b4ff4a9eed651303b1efa` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.265131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e68b72cd3b4ff4a9eed651303b1efa)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.271306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.329295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3246ceeb7d64de8b095f5d7c305b7c4', 'request_id': 'req-f77eabdf-3cbd-447b-98e0-521c200f883b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.347947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3246ceeb7d64de8b095f5d7c305b7c4', 'request_id': 'req-f77eabdf-3cbd-447b-98e0-521c200f883b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.357347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3246ceeb7d64de8b095f5d7c305b7c4', 'request_id': 'req-f77eabdf-3cbd-447b-98e0-521c200f883b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.375619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] The token cache is being invalidate because project b3246ceeb7d64de8b095f5d7c305b7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:14.376093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.376875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.379086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 963/1920] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:13 2022] DELETE /identity/v3/projects/b3246ceeb7d64de8b095f5d7c305b7c4 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:14.380951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.383497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.389228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e68b72cd3b4ff4a9eed651303b1efa'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.390293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e68b72cd3b4ff4a9eed651303b1efa'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.399393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e68b72cd3b4ff4a9eed651303b1efa'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.421397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e68b72cd3b4ff4a9eed651303b1efa'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.447222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ce658c9a-9dde-40fd-b0df-953640c985b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e68b72cd3b4ff4a9eed651303b1efa'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:14.472715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 958/1921] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:14 2022] DELETE /identity/v3/users/21e68b72cd3b4ff4a9eed651303b1efa => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:14.474477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb595-0368-4e06-8e38-82444539822f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.475280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb595-0368-4e06-8e38-82444539822f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.475768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb595-0368-4e06-8e38-82444539822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.501613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eabdf-3cbd-447b-98e0-521c200f883b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.511037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bd1fb595-0368-4e06-8e38-82444539822f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:14.512837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bd1fb595-0368-4e06-8e38-82444539822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.596594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.598355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.598677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.600215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.604969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.617696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 959/1922] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:14.625914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.626275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.626563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.656277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.657818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.667521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:14.668635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 964/1923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:14.669160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:14.679145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 960/1924] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:14.683727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.696312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce85-ce00-4127-8990-2c76801b1e49 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.701697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.702333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.702617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.702893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.704177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:14.704463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.717108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.745798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.752791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 965/1925] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:14.760045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72453c-24a7-48e5-9306-b7031e08ded8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.766565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.767356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.767741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.768122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.769184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.771962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.819736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 961/1926] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:14 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:14.824754 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.836970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e8a20-bd10-45ac-8138-8e53b2fb97be admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.841541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:14.918735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.919568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.920023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.920425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.921339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.923824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.960697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 966/1927] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:14 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:14.960975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:14.973155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb3a4d-bbf6-48d8-8611-c80011c5a9aa admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:14.978251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:14.979009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:14.979287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:14.979661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:14.980691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:14.982141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:14.983679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:14.999852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 962/1928] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:15.001724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.025759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536eb87-b363-4f20-986d-3a41a6b93904 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.032051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.033063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.033063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.033256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.033892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.035831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475e54e-01dd-4b59-bc63-165e1695e875 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.051801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 967/1929] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:15.052706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.063271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21166776-2ff5-4af4-aef7-0e8fe6eff1e6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.070968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.071925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.072410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.072872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] PATH_INFO: `/v3/projects/1a2a6c176bff412890595e8afca5f583/users/f35147eca46144d2909290e970d37d6c/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.092885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2a6c176bff412890595e8afca5f583, user_id=f35147eca46144d2909290e970d37d6c, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.096655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.112388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 963/1930] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:15 2022] PUT /identity/v3/projects/1a2a6c176bff412890595e8afca5f583/users/f35147eca46144d2909290e970d37d6c/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:15.126251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.126547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.126906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.184710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] MFA Rules not processed for user `f35147eca46144d2909290e970d37d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:15.186183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.334075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 968/1931] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:15.339053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.473901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.475361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.485696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8609750-424b-4558-854e-c09f86c7817f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.496975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2f44-4f94-4bcd-8eaf-d1ddf08d0df1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.499381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.500277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.500786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.501260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.502145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.505271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.506919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.507722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.508176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.508528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] PATH_INFO: `/v3/projects/c99b430909f64f229eb8f603afb52530` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.510710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99b430909f64f229eb8f603afb52530)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.512867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.557616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 964/1932] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:15 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:15.561992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.564265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.574454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99b430909f64f229eb8f603afb52530', 'request_id': 'req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.584409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99b430909f64f229eb8f603afb52530', 'request_id': 'req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.591688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99b430909f64f229eb8f603afb52530', 'request_id': 'req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.601603 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:15.607525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] The token cache is being invalidate because project c99b430909f64f229eb8f603afb52530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:15.608263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.609241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.612297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 969/1933] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:15 2022] DELETE /identity/v3/projects/c99b430909f64f229eb8f603afb52530 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:15.612858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.615842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.636516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed026-8a33-4bc7-8c30-ac804b02aaf6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.641504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.642189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.642470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.642772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] PATH_INFO: `/v3/projects/7e67b54b9de24a65b9d350d9a99bd1f9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.643869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e67b54b9de24a65b9d350d9a99bd1f9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.646002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.713400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e67b54b9de24a65b9d350d9a99bd1f9', 'request_id': 'req-bc092223-513c-46e4-afde-f80e76393fdd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.714350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445429a-ba1d-4227-b47a-68a1ab8d1b2c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.723256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e67b54b9de24a65b9d350d9a99bd1f9', 'request_id': 'req-bc092223-513c-46e4-afde-f80e76393fdd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.728857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:15.731931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e67b54b9de24a65b9d350d9a99bd1f9', 'request_id': 'req-bc092223-513c-46e4-afde-f80e76393fdd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.767007 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] The token cache is being invalidate because project 7e67b54b9de24a65b9d350d9a99bd1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:15.767349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.768775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bc092223-513c-46e4-afde-f80e76393fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:15.770799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 965/1934] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:15 2022] DELETE /identity/v3/projects/7e67b54b9de24a65b9d350d9a99bd1f9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:15.772134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538c178-842d-4607-815c-c7a425398383 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.772418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538c178-842d-4607-815c-c7a425398383 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.772689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538c178-842d-4607-815c-c7a425398383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.812524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f538c178-842d-4607-815c-c7a425398383 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:15.816336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f538c178-842d-4607-815c-c7a425398383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.872916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.873819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.874251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.874697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.876696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.878868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.908511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 966/1935] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:15.909515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538c178-842d-4607-815c-c7a425398383 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.910310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 970/1936] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:15 2022] POST /identity/v3/users => generated 296 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:15.912105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f538c178-842d-4607-815c-c7a425398383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.912232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:15.913475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:15.929198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538c178-842d-4607-815c-c7a425398383 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.932469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7b88-6d3e-458e-9169-fd02984a8e58 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:15.936016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.936664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.937044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.937320 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] PATH_INFO: `/v3/users/cab16c35892d410e8556becc422364ad` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.937428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:15.939176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab16c35892d410e8556becc422364ad)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.939293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.939481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.939768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.940420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:15.941820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.942057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ed0-75ce-4e75-b07b-b36dc4a1ca2a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:15.957991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 971/1937] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:15.959620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:15.959911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:15.960190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:15.994121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:15.995406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.007327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 972/1938] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:16.008199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.019965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d2fa5-b2b4-48ad-8357-a3cf13aff252 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.023534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab16c35892d410e8556becc422364ad'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.024544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab16c35892d410e8556becc422364ad'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.025292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab16c35892d410e8556becc422364ad'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.025905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.026674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.026961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.027238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] PATH_INFO: `/v3/projects/0ead8df2691f4c84af016e01a627ee8f/users/c0a983f41acb47d48a3f99562b062ec1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.034921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab16c35892d410e8556becc422364ad'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.042879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ead8df2691f4c84af016e01a627ee8f, user_id=c0a983f41acb47d48a3f99562b062ec1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.045752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.050106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab16c35892d410e8556becc422364ad'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.060806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 973/1939] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:16 2022] PUT /identity/v3/projects/0ead8df2691f4c84af016e01a627ee8f/users/c0a983f41acb47d48a3f99562b062ec1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:16.062009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.065847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.067875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 967/1940] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:15 2022] DELETE /identity/v3/users/cab16c35892d410e8556becc422364ad => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:16.073426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.073565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.153925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:16.167677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a2d9-c6cd-42bd-b215-31e7a132d54a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.175474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.176468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.177019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.177605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.178642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.180627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.182534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:16.188105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430dcaf5-d03e-49d3-8514-ab8dd6814041 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.204049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.204049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.204049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.204367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.204870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:16.205938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 968/1941] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:16.206063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.207164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.211916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.220490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea5cbe-366b-4a5a-af34-9a98dee069e1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.226692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.227809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.228402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.228864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] PATH_INFO: `/v3/projects/0ead8df2691f4c84af016e01a627ee8f/users/c0a983f41acb47d48a3f99562b062ec1/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.231437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ead8df2691f4c84af016e01a627ee8f, user_id=c0a983f41acb47d48a3f99562b062ec1, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.235089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c99b0-8c5a-40ea-880d-f03b8e260944 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.241518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 974/1942] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:16.242945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.261338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31956236-0abd-4853-9d42-dc91beb876fb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.261563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 969/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:16 2022] PUT /identity/v3/projects/0ead8df2691f4c84af016e01a627ee8f/users/c0a983f41acb47d48a3f99562b062ec1/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:16.265388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.265793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.270675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.315079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] MFA Rules not processed for user `c0a983f41acb47d48a3f99562b062ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:16.316463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.389436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.390590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.391477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.393049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.394120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.400100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.443623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 975/1944] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:16 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:16.448115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.454962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 970/1945] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:16.463322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.465203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.577456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b073fa5-6f88-49d2-85ca-fafc509fa3ba admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.596968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.597615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.598260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.598551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.598941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1ab4f-18aa-4bfa-b9bb-9ce4806a94cb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.599183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.601636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.634297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 976/1946] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:16 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:16.634297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.637153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.656045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36044-3f7d-477c-947a-6375e6b88a1c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:16.662489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.663354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.663693 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.664007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] PATH_INFO: `/v3/projects/78b2653c21074cd7b547a5099ec086f5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.665224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b2653c21074cd7b547a5099ec086f5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.668376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.670406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:16.671116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:16.671527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:16.672456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:16.749078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b2653c21074cd7b547a5099ec086f5', 'request_id': 'req-8eac697c-8a5b-46af-a511-56b9b2b4ede5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.772267 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b2653c21074cd7b547a5099ec086f5', 'request_id': 'req-8eac697c-8a5b-46af-a511-56b9b2b4ede5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.777178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:16.780834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:16.797879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b2653c21074cd7b547a5099ec086f5', 'request_id': 'req-8eac697c-8a5b-46af-a511-56b9b2b4ede5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.827832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] The token cache is being invalidate because project 78b2653c21074cd7b547a5099ec086f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:16.828132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.830440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:16.837142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 977/1947] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:16 2022] DELETE /identity/v3/projects/78b2653c21074cd7b547a5099ec086f5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:16.838090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.842065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:16.852725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:16.861624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:16.884897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 971/1948] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:16.890404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:16.907528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a1489-fcab-446b-a8f8-018f0a97de1e service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.020939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac697c-8a5b-46af-a511-56b9b2b4ede5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.033918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.034749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.036025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.036602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.037097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.038237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.038527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.038807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] PATH_INFO: `/v3/users/107ea84531704f9d83fc307d1b209fa0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.040013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.041161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=107ea84531704f9d83fc307d1b209fa0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.042543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.044020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.079553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 978/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:17.085508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.113581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc788e-67f1-48d8-9664-1fe80c125bf5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.121452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.122196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.122496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.122983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.123679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.125588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.144478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 979/1950] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:17.149354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.174980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea68ce-fb3c-4229-91da-7d97250300b7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.175512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107ea84531704f9d83fc307d1b209fa0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.176601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107ea84531704f9d83fc307d1b209fa0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.177644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107ea84531704f9d83fc307d1b209fa0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.183730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.187359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.189769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.190084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] PATH_INFO: `/v3/projects/49cbbf2c39a84f71a78bdafa4b22782e/users/c5ad6fa600fc4eecb6ffa3954d786329/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.190903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107ea84531704f9d83fc307d1b209fa0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.221784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cbbf2c39a84f71a78bdafa4b22782e, user_id=c5ad6fa600fc4eecb6ffa3954d786329, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.223534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107ea84531704f9d83fc307d1b209fa0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.225169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845650-7394-42d3-8eb2-809c6ce60fb5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.242549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 980/1951] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:17 2022] PUT /identity/v3/projects/49cbbf2c39a84f71a78bdafa4b22782e/users/c5ad6fa600fc4eecb6ffa3954d786329/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:17.248016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.254467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.254804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.269749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 972/1952] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:16 2022] DELETE /identity/v3/users/107ea84531704f9d83fc307d1b209fa0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:17.292316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:17.293666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:17.401211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 981/1953] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:17.414873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.416907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:17.491878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.493346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:17.495379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18670385-4f4d-47db-96a4-864cb70638b8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.500184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.500843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.501126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.501401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.512170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:17.524993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.530231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.543633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 973/1954] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:17.557397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:17.580981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dff16-ad15-45c6-8717-38ca8112d719 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.586405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0a983f41acb47d48a3f99562b062ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ead8df2691f4c84af016e01a627ee8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.587148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.587558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.588047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.588812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:17.589244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.591125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.605764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 982/1955] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:17.608956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.620302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76847e6b-2236-47b0-bb3f-f77e1e6cbe27 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.627777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0a983f41acb47d48a3f99562b062ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ead8df2691f4c84af016e01a627ee8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.628132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.628560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.628848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.629554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.631629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.665747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 974/1956] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:17 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:17.667698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.679816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939bf6c-6e34-4cea-9c62-328ab98537f3 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.753101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0a983f41acb47d48a3f99562b062ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ead8df2691f4c84af016e01a627ee8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.753769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.754031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.754327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.755157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:17.755469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.757231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.773888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 983/1957] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:17.818382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:17.830289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f704de8-4072-49cb-961d-4560a319e0a1 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:17.835496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0a983f41acb47d48a3f99562b062ec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ead8df2691f4c84af016e01a627ee8f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:17.836399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:17.836893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:17.837401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] PATH_INFO: `/v3/projects/6b824001930c4e8796ff362325867e1d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:17.838781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6b824001930c4e8796ff362325867e1d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:17.841346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:17.900885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b824001930c4e8796ff362325867e1d', 'request_id': 'req-77542c47-1ba0-46cd-921a-3b032de94cb6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.909626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b824001930c4e8796ff362325867e1d', 'request_id': 'req-77542c47-1ba0-46cd-921a-3b032de94cb6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.916105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b824001930c4e8796ff362325867e1d', 'request_id': 'req-77542c47-1ba0-46cd-921a-3b032de94cb6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.939842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] The token cache is being invalidate because project 6b824001930c4e8796ff362325867e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:17.940176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.940920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-77542c47-1ba0-46cd-921a-3b032de94cb6 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:17.943301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 975/1958] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:17 2022] DELETE /identity/v3/projects/6b824001930c4e8796ff362325867e1d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:18.225263 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.227194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.322127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e303df-ee3a-4780-83a1-3dec1faddbf0 tempest-VolumeQuotaClassesTest-800392880 tempest-VolumeQuotaClassesTest-800392880-project-admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:18.345620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.346407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.346691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.415043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] MFA Rules not processed for user `c5ad6fa600fc4eecb6ffa3954d786329`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:18.417516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.463196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:18.471310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.471310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.471516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] PATH_INFO: `/v3/users/f35147eca46144d2909290e970d37d6c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.476041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35147eca46144d2909290e970d37d6c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:18.485224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:18.540932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:18.542034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 976/1959] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:18.548751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.550383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.570437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e063f0-a440-48dc-9b63-36ea10a76758 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:18.589825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:18.590869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.591277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.591626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.592945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:18.596610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:18.597721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35147eca46144d2909290e970d37d6c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.598566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35147eca46144d2909290e970d37d6c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.599295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35147eca46144d2909290e970d37d6c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.619803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35147eca46144d2909290e970d37d6c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.637213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 977/1960] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:18 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:18.642695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.644760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35147eca46144d2909290e970d37d6c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.645142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.676534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 984/1961] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:18 2022] DELETE /identity/v3/users/f35147eca46144d2909290e970d37d6c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:18.677574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.679114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.688238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:18.748626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049cfee-fde3-4e86-8e26-c69d607c854a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:18.762922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad68fe3-4554-48d9-a3ee-f7d2c861224b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:18.768823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:18.769530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.769907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.770257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.771038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:18.773033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a7147-f7d5-45a3-9e87-d102f9fc1407 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:18.799414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 985/1962] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:18 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:18.799414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.799414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.799718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.816051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:18.817266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.817561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.817996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] PATH_INFO: `/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.819465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0d9c2290ac48a19b5737faa8ec7864)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:18.821384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:18.822225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:18.822628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.831016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 986/1963] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:18.831815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.844755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66bc8d-88cb-4a9a-a792-697d1720b806 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:18.849387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:18.850040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:18.850329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:18.850601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:18.851247 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:18.852846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:18.865751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 987/1964] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:18.865751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.868985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.883591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0d9c2290ac48a19b5737faa8ec7864', 'request_id': 'req-3e302f86-d273-4ffc-8732-e02d43bca4c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.893602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0d9c2290ac48a19b5737faa8ec7864', 'request_id': 'req-3e302f86-d273-4ffc-8732-e02d43bca4c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.903997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0d9c2290ac48a19b5737faa8ec7864', 'request_id': 'req-3e302f86-d273-4ffc-8732-e02d43bca4c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.941609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] The token cache is being invalidate because project 0d0d9c2290ac48a19b5737faa8ec7864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:18.942049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.942998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:18.944953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 978/1965] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:18 2022] DELETE /identity/v3/projects/0d0d9c2290ac48a19b5737faa8ec7864 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:18.946551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:18.948245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:18.952808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0bc6-364c-407e-9e9c-2ec62fc12c65 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.002284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:19.034870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.035689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.036069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.036447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.038702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:19.042224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302f86-d273-4ffc-8732-e02d43bca4c8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.047964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.048539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.048818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.049092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] PATH_INFO: `/v3/projects/d1e9922d50f0452ab1a4bee0543f4773/users/1a763bc01b4049529949ee8c788e435f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.054981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:19.056396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.058707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.072481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e9922d50f0452ab1a4bee0543f4773, user_id=1a763bc01b4049529949ee8c788e435f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.076799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86f80a-e05f-4764-baeb-6a8e06738fa5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.098212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 979/1966] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:18 2022] PUT /identity/v3/projects/d1e9922d50f0452ab1a4bee0543f4773/users/1a763bc01b4049529949ee8c788e435f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:19.100537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 988/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:19.103054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7e80-a84c-4ef2-bd4d-4a8733febf06 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.103356 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7e80-a84c-4ef2-bd4d-4a8733febf06 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.103707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7e80-a84c-4ef2-bd4d-4a8733febf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.103839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.124560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094c41-dfd7-46c7-b2f6-6c3f7e9b274c service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.128944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-669c7e80-a84c-4ef2-bd4d-4a8733febf06 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:19.130368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-669c7e80-a84c-4ef2-bd4d-4a8733febf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:19.214220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 980/1968] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:19.224579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.225058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.225500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.240842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.242316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.242613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.243031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.243701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.245347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.252521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:19.254121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:19.261204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 989/1969] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:19.265135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.269282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 981/1970] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:19.273605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.279255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87374-6b0f-46b6-93af-2fddf9c69232 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.283677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e288b-da5c-4de0-9a7b-beb641dae988 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.286863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.288144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.288488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.288776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] PATH_INFO: `/v3/projects/d1e9922d50f0452ab1a4bee0543f4773/users/1a763bc01b4049529949ee8c788e435f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.289847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801c570-daec-4381-a314-cf990356603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.290628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801c570-daec-4381-a314-cf990356603f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.290925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e9922d50f0452ab1a4bee0543f4773, user_id=1a763bc01b4049529949ee8c788e435f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.291257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801c570-daec-4381-a314-cf990356603f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.291768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801c570-daec-4381-a314-cf990356603f admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.292644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801c570-daec-4381-a314-cf990356603f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:19.293102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801c570-daec-4381-a314-cf990356603f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.293910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.295053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801c570-daec-4381-a314-cf990356603f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.313599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 990/1971] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:19 2022] PUT /identity/v3/projects/d1e9922d50f0452ab1a4bee0543f4773/users/1a763bc01b4049529949ee8c788e435f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:19.314327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 982/1972] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:19.314986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.421536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332ed69-1bdb-44a0-b62b-ef64779c383b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.427572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.428279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.428630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.428989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.429675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.431842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.475547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 991/1973] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:19 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:19.479872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801c570-daec-4381-a314-cf990356603f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.493988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801c570-daec-4381-a314-cf990356603f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.599664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.600716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.601880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.602416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.603399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.604602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.606574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.608424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:19.628622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5b813-9bd3-44d2-81fd-ffbe3d9a7c8f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.643375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 983/1974] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:19 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:19.644942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.652094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.655700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.656009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.659012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] PATH_INFO: `/v3/projects/1a2a6c176bff412890595e8afca5f583` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.661845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2a6c176bff412890595e8afca5f583)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.663222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4411fc-cfc3-4564-a52e-9892b24108a8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.666097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.670370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.671455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.671897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.672634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.673417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.676307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.695935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 984/1975] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:19.700600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.713502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b0995-fcc9-4dbc-b6ea-fb9ed1c9f27d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.721574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.722491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.723732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.724167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.725031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.727501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.744389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 985/1976] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:19.749286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.765257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:19.768662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c3f69-dec7-488b-9daa-e1a9e0df13ce admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:19.793969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2a6c176bff412890595e8afca5f583', 'request_id': 'req-3b0574a4-ceab-4e7f-bc02-386da8349d30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:19.816527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2a6c176bff412890595e8afca5f583', 'request_id': 'req-3b0574a4-ceab-4e7f-bc02-386da8349d30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:19.821495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:19.835427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2a6c176bff412890595e8afca5f583', 'request_id': 'req-3b0574a4-ceab-4e7f-bc02-386da8349d30'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:19.852564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] The token cache is being invalidate because project 1a2a6c176bff412890595e8afca5f583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:19.852981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:19.853752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:19.859165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 992/1977] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:19 2022] DELETE /identity/v3/projects/1a2a6c176bff412890595e8afca5f583 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:19.861196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:19.864004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:19.888325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:19.888933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.889371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.889657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] PATH_INFO: `/v3/projects/9b6d13c010304755ba81ce8f31459b87/users/10a5c8ccdc2b49d68a6c088207d593d3/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.913812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6d13c010304755ba81ce8f31459b87, user_id=10a5c8ccdc2b49d68a6c088207d593d3, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:19.917244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da41ed9-f04b-4a27-a28f-70ac8456c204 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:19.940289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 986/1978] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:19 2022] PUT /identity/v3/projects/9b6d13c010304755ba81ce8f31459b87/users/10a5c8ccdc2b49d68a6c088207d593d3/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:19.943002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:19.943394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:19.943786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:19.999021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:20.004702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.030023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0574a4-ceab-4e7f-bc02-386da8349d30 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:20.118474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:20.120717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:20.121502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:20.122058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] PATH_INFO: `/v3/users/c0a983f41acb47d48a3f99562b062ec1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:20.124419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a983f41acb47d48a3f99562b062ec1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:20.128670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:20.160787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 987/1979] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:20.171017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:20.172788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.246121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a983f41acb47d48a3f99562b062ec1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:20.247077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a983f41acb47d48a3f99562b062ec1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:20.247886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a983f41acb47d48a3f99562b062ec1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:20.259081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a983f41acb47d48a3f99562b062ec1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:20.274229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:20.276281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-911207ab-488e-4ecf-b4b2-3ca753211071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a983f41acb47d48a3f99562b062ec1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:20.280040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27219d2-e632-42e3-ab39-f5317abdf340 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:20.289942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:20.291719 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:20.292163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:20.292603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:20.295540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.307414 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 993/1980] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:19 2022] DELETE /identity/v3/users/c0a983f41acb47d48a3f99562b062ec1 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:20.323738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:20.325204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:20.339800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 988/1981] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:20.502794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:20.503249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:20.503770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:20.548444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] MFA Rules not processed for user `1a763bc01b4049529949ee8c788e435f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:20.550451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.658206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 994/1982] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:20.665663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:20.667381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.785144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc84c5-9b18-43f6-8e0d-daaed13cdbd0 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:20.808710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:20.824658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:20.826686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.883321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:20.884388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:20.884925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:20.885466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:20.887567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:20.946217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed24438-97ce-425c-b758-a3ff63b8e6c8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:20.951997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:20.952669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:20.952960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:20.953244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] PATH_INFO: `/v3/projects/0ead8df2691f4c84af016e01a627ee8f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:20.954552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ead8df2691f4c84af016e01a627ee8f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:20.957515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:20.986962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:20.988443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d1620-2686-4535-96d2-8e85e15f8a11 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:20.999587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 989/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:21.015503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ead8df2691f4c84af016e01a627ee8f', 'request_id': 'req-56dc274a-d358-4ec6-9ebf-38c38b92d369'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:21.029744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ead8df2691f4c84af016e01a627ee8f', 'request_id': 'req-56dc274a-d358-4ec6-9ebf-38c38b92d369'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:21.039476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ead8df2691f4c84af016e01a627ee8f', 'request_id': 'req-56dc274a-d358-4ec6-9ebf-38c38b92d369'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:21.055447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] The token cache is being invalidate because project 0ead8df2691f4c84af016e01a627ee8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:21.056130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:21.057121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-56dc274a-d358-4ec6-9ebf-38c38b92d369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:21.059593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 995/1984] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:20 2022] DELETE /identity/v3/projects/0ead8df2691f4c84af016e01a627ee8f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:21.064322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41bb7-7f77-45fb-8776-0d4fb2302102 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.064686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41bb7-7f77-45fb-8776-0d4fb2302102 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.065574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41bb7-7f77-45fb-8776-0d4fb2302102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.091801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-29b41bb7-7f77-45fb-8776-0d4fb2302102 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:21.093041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-29b41bb7-7f77-45fb-8776-0d4fb2302102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.191886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 990/1985] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:21.193926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.194906 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.195437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.212742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:21.249652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:21.257751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.274544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.274905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.275108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.301894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 996/1986] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:21.306617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.318232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] MFA Rules not processed for user `10a5c8ccdc2b49d68a6c088207d593d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:21.319331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.327426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c0136-97b1-47e0-952f-4c544a709f82 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.343122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.344974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.346943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.347539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.349195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:21.350313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.355304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.370606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 997/1987] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:21.374750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.391875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55389a4-1751-4ba6-84f5-1f5f9eefa9f1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.400383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.401250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.401721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.402151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.402961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.405147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.417662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:21.423386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 991/1988] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:21.436097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.438637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.462520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 998/1989] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:21.466128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.519627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084cc5a-eba7-4b44-83a8-29419442c48f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.581584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdbf82-e261-4ad9-ba84-48332509bfe5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.586213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.586857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.587207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.587499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.588103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.588723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.589004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.589320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.589978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.596134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.627903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 999/1990] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:21.628863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.638522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289af9e-366d-44ef-b39f-f497fee0de5c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.644290 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.645139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.645557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.645953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.646601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.648184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.651346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.652796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.662368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1000/1991] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:21.662638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 992/1992] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:21.665327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.677244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc187ab-440c-40d0-85d1-49d60729481d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.682675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.683396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.683698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.683944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.684594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.686213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2846bd-2087-48e6-b4ff-8eca2818b230 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.703933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1001/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:21.705206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.716631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf6eb5-c7da-4fe3-a682-57b9944dc084 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.721990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.722722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.723008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.723360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] PATH_INFO: `/v3/projects/5117a004453e441c98630912c0818549/users/8f1b5a03715c4fa8b6851a11e4c2ffdb/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.740342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5117a004453e441c98630912c0818549, user_id=8f1b5a03715c4fa8b6851a11e4c2ffdb, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.743120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:21.757535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 993/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:21 2022] PUT /identity/v3/projects/5117a004453e441c98630912c0818549/users/8f1b5a03715c4fa8b6851a11e4c2ffdb/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:21.769719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.770022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.770305 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.797917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] MFA Rules not processed for user `8f1b5a03715c4fa8b6851a11e4c2ffdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:21.799269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.874498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1002/1995] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:21.885647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:21.887767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:21.959334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:21.966354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206ae0-c53d-4eda-8580-cf97b36b5693 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:21.972747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:21.973895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:21.974244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:21.974588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:21.990402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:21.998582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d7f44-fb9b-47d0-9e96-eeec03ab45ac service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:22.017096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 994/1996] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:22.606549 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:22.616358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0644d-941b-4082-ac3d-ef47368a2dd3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:22.619948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:22.620544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:22.620830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:22.621105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:22.622860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:22.640110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:22.686417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:22.687885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:22.697993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1003/1997] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:23.073068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673662a7-add0-4939-9790-ee1ed8094b67 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:23.073660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673662a7-add0-4939-9790-ee1ed8094b67 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:23.073660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673662a7-add0-4939-9790-ee1ed8094b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:23.093526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-673662a7-add0-4939-9790-ee1ed8094b67 None None] MFA Rules not processed for user `10a5c8ccdc2b49d68a6c088207d593d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:23.094811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-673662a7-add0-4939-9790-ee1ed8094b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:23.152165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 995/1998] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:30.509876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:30.589333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fc351-7b73-4c60-b839-3225c5f69ea4 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:30.596655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:30.597387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:30.597774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:30.598179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] PATH_INFO: `/v3/users/8f1b5a03715c4fa8b6851a11e4c2ffdb` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:30.599589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1b5a03715c4fa8b6851a11e4c2ffdb)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:30.601512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:30.675031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1b5a03715c4fa8b6851a11e4c2ffdb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:30.675761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1b5a03715c4fa8b6851a11e4c2ffdb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:30.676422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1b5a03715c4fa8b6851a11e4c2ffdb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:30.683929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1b5a03715c4fa8b6851a11e4c2ffdb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:30.699897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1b5a03715c4fa8b6851a11e4c2ffdb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:30.717914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1004/1999] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:30 2022] DELETE /identity/v3/users/8f1b5a03715c4fa8b6851a11e4c2ffdb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:30.720191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:30.720659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:30.720778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:30.742606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:30.743876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:30.819668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 996/2000] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:30.836139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:30.837451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:30.885820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:30.913980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b0956-7ce2-4f6d-a7a4-0fc2fe7056f7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:30.918573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:30.919193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:30.919482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:30.919758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:30.930218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:30.931679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6afe88-f6b9-40dd-97f2-08fc558eacba service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:30.943651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1005/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:31.561132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:31.563002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:31.630223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a9bf8-9593-4ace-a031-0d26f2e6a87c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:31.644431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:31.645136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:31.645418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:31.645692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] PATH_INFO: `/v3/projects/5117a004453e441c98630912c0818549` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:31.646947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5117a004453e441c98630912c0818549)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:31.649383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:31.724441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5117a004453e441c98630912c0818549', 'request_id': 'req-197fbc97-a436-479f-a02d-aaea80abd0c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:31.732645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5117a004453e441c98630912c0818549', 'request_id': 'req-197fbc97-a436-479f-a02d-aaea80abd0c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:31.740526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5117a004453e441c98630912c0818549', 'request_id': 'req-197fbc97-a436-479f-a02d-aaea80abd0c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:31.757682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] The token cache is being invalidate because project 5117a004453e441c98630912c0818549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:31.758090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:31.758752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-197fbc97-a436-479f-a02d-aaea80abd0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:31.760868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 997/2002] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:31 2022] DELETE /identity/v3/projects/5117a004453e441c98630912c0818549 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:31.767730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:31.768207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:31.768297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:31.789369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:31.790806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:31.872410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1006/2003] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:31.884427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:31.885937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:31.886499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:31.941276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:31.944175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:31.968874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 998/2004] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:31.970877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:31.991053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe411a6-8cd0-49e5-98ae-ba8678ee7a04 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.010213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.011268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.011869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.014666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.017343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:32.017343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.024492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.058110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1007/2005] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:32.065045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.084978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15e25c-50c4-4c01-beb8-64d11f77390d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.095770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.097108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.098097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.098769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.100175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.104422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.125074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:32.168199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 999/2006] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:32.175691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.193637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c04065-a0ec-4889-a995-3e5b10532220 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.289798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.290653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.291367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.291713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.292428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.294644 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.327956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1008/2007] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:32.333034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.346434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f6bf6-7724-4f4f-945f-35983949eb8d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.357278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.358095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.358359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.358639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.359297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.361433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.383201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1000/2008] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:32.383339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.395112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a40897-b7f0-4958-b7ee-9925c2cb949c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.400771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.401395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.401681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.401990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.402725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.404233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29576dfb-f84d-409a-b5b3-be4ce7889e66 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.416119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1009/2009] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:32.420474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.430683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359a63-8db4-4c34-8e2c-ba8ac718062d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.435395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.436153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.436509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.436858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] PATH_INFO: `/v3/projects/923fa2148c934284a04514ceda1b227e/users/1cb5b1eb9c784fd38266ac9851ade54e/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.453239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=923fa2148c934284a04514ceda1b227e, user_id=1cb5b1eb9c784fd38266ac9851ade54e, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.457110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.473784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1001/2010] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:32 2022] PUT /identity/v3/projects/923fa2148c934284a04514ceda1b227e/users/1cb5b1eb9c784fd38266ac9851ade54e/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:32.482763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.483055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.483334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.513572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] MFA Rules not processed for user `1cb5b1eb9c784fd38266ac9851ade54e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:32.514887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:32.545790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:32.580757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1010/2011] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:32.585263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.649291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068a01b-389f-47df-8fc0-96fce9e6ad18 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.654050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.654655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.654940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.655255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.655850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.657856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.685066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1002/2012] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:32.689610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.700593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db773f7-fbd7-4e6d-ba65-34f8fd76b8aa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.761452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.762092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.762376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.762652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.763286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.765206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.786041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1011/2013] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:32.790040 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.800532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46209a1-b69c-4511-a8a7-ef5653526ed7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.805275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.805931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.806241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.806494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.807133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.808770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.819972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1003/2014] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:32.824085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.834987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3709-b7a8-41c9-8c4c-83a50664834d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.840582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.841197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.841478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.841784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.842426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.844442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce25e6-b510-4818-af0c-c63dcc6f0ac9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.858623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1012/2015] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:32.860849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:32.871304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee3859-5de7-4b87-bb5d-2e620454aa67 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:32.877054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:32.877738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.878066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.878339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] PATH_INFO: `/v3/projects/37b28854b3524a8e9f77cad66f5a1d88/users/9de8a27c2f224a15ab1a9aa4c7a6da93/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.902800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b28854b3524a8e9f77cad66f5a1d88, user_id=9de8a27c2f224a15ab1a9aa4c7a6da93, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:32.905720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:32.921781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1004/2016] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:32 2022] PUT /identity/v3/projects/37b28854b3524a8e9f77cad66f5a1d88/users/9de8a27c2f224a15ab1a9aa4c7a6da93/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:32.932723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:32.933392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:32.933392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:32.964772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] MFA Rules not processed for user `9de8a27c2f224a15ab1a9aa4c7a6da93`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:32.966034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:33.053018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1013/2017] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:33.057561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.132254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802abd8-524c-4035-be5d-d28b5abe63e5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.136907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.137500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.137818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.138072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.138698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.140704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.151104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:33.173748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1005/2018] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:33 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:33.177670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.188816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ac5c7-8088-4038-a87f-aa85ca77c60a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.231477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:33.250738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.251343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.251623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.251897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.252518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.254474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.275398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1014/2019] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:33 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:33.279258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.292282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d7641-794e-4090-91b1-2f4cf290400c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.297221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.297878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.298171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.298451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.299101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.300725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.310714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1006/2020] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:33.314975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.325408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465f081-1d41-4d63-8b6e-247b158b7237 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.330917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.331319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.331598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.331887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] PATH_INFO: `/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a/users/55ce7c30cfb84f20b469f06abfbaa5cb/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.348553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e9893d64a44d93bb79ae591a84ed4a, user_id=55ce7c30cfb84f20b469f06abfbaa5cb, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.351390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.366903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1015/2021] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:33 2022] PUT /identity/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a/users/55ce7c30cfb84f20b469f06abfbaa5cb/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:33.371189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.383004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a98663-1ae1-4a93-998a-edb55b17c9a8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.451639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.452284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.452932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.452932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.454119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.455961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f5ee3-3321-40d8-a639-50d2f1fe0b7f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.468289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1007/2022] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:33.472986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.484357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2915f-54d5-43b6-beb3-634dcc6f454d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.490140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.490959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.491235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.491496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] PATH_INFO: `/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a/users/55ce7c30cfb84f20b469f06abfbaa5cb/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.493895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e9893d64a44d93bb79ae591a84ed4a, user_id=55ce7c30cfb84f20b469f06abfbaa5cb, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.496809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.517195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1016/2023] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:33 2022] PUT /identity/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a/users/55ce7c30cfb84f20b469f06abfbaa5cb/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:33.519865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.520045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.520260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.556501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] MFA Rules not processed for user `55ce7c30cfb84f20b469f06abfbaa5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:33.557773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:33.626087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1008/2024] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:33.637427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:33.639134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:33.696208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016742a-5a03-4974-8be1-3174696e05fe admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:33.699701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:33.700298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:33.700576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:33.700846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:33.754987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:33.756420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:33.765335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1017/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:35.316932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:35.318399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:35.330394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f15f6-64e8-4591-9904-3fbeeb0423d1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:35.333947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:35.334560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:35.335527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:35.335813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:35.395070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:35.396533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b369ab-94bc-43d3-a936-a1cdf51de57a service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:35.400163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:35.409511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1009/2026] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:36.095717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:36.108097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5aaa9-a1d6-47f5-9136-1b65d434e302 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:36.113196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:36.114129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:36.114483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:36.114896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:36.126973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:36.128837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60272-bb44-4c72-9ba8-5f0641160706 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:36.142489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1018/2027] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:38.348029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:38.348910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:38.349189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:38.350633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1010/2028] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:38:38.384834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:38.385286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:38.385558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:38.406342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] MFA Rules not processed for user `45b541633f92420e9b118c25775d646a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:38.408521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:38.441187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:38.477460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1019/2029] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:38.489760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:38.491535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:38.509118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594359f-7ee2-47f8-a037-7afe8c55801c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:38.514467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:38.515177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:38.515529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:38.515815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:38.525123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:38.526819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:38.537465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1011/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:39.486088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:39.488323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:39.584033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47ad8-c2c7-4b5c-b92d-fe276c1f94d1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:39.589339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:39.590616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:39.591104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:39.591590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] PATH_INFO: `/v3/users/c5ad6fa600fc4eecb6ffa3954d786329` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:39.592946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ad6fa600fc4eecb6ffa3954d786329)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:39.595810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:39.653922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ad6fa600fc4eecb6ffa3954d786329'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:39.654587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ad6fa600fc4eecb6ffa3954d786329'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:39.655218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ad6fa600fc4eecb6ffa3954d786329'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:39.663845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ad6fa600fc4eecb6ffa3954d786329'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:39.677224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ad6fa600fc4eecb6ffa3954d786329'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:39.690729 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1020/2031] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:39 2022] DELETE /identity/v3/users/c5ad6fa600fc4eecb6ffa3954d786329 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:39.869358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:39.884622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ca35d-b807-4e0c-b49f-f51794d4e960 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:39.951618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:39.952373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:39.952754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:39.953246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] PATH_INFO: `/v3/projects/49cbbf2c39a84f71a78bdafa4b22782e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:39.954628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49cbbf2c39a84f71a78bdafa4b22782e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:39.956915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:39.960846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:39.962970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:39.977887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e45161-512b-487c-b215-0b10261f0f27 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:39.983030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:39.984399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:39.985229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:39.985503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] PATH_INFO: `/v3/users/a1a9b41251e146eba110fa1e2b1fc9c5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:39.986848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a9b41251e146eba110fa1e2b1fc9c5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:39.989117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.009682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49cbbf2c39a84f71a78bdafa4b22782e', 'request_id': 'req-d289f82f-4469-4a33-9ffe-d05e70415593'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.017730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49cbbf2c39a84f71a78bdafa4b22782e', 'request_id': 'req-d289f82f-4469-4a33-9ffe-d05e70415593'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.025495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49cbbf2c39a84f71a78bdafa4b22782e', 'request_id': 'req-d289f82f-4469-4a33-9ffe-d05e70415593'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.044052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:40.044725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] The token cache is being invalidate because project 49cbbf2c39a84f71a78bdafa4b22782e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:40.045027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.045791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.047883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1012/2032] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:39 2022] DELETE /identity/v3/projects/49cbbf2c39a84f71a78bdafa4b22782e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:40.050846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a9b41251e146eba110fa1e2b1fc9c5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.050846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a9b41251e146eba110fa1e2b1fc9c5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.050846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a9b41251e146eba110fa1e2b1fc9c5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.051919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:40.053464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.058118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a9b41251e146eba110fa1e2b1fc9c5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.071165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a9b41251e146eba110fa1e2b1fc9c5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.087169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1021/2033] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:39 2022] DELETE /identity/v3/users/a1a9b41251e146eba110fa1e2b1fc9c5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:40.135160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f82f-4469-4a33-9ffe-d05e70415593 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:40.155481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:40.210478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:40.211140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.211816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.211816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] PATH_INFO: `/v3/users/1a763bc01b4049529949ee8c788e435f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.212862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a763bc01b4049529949ee8c788e435f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:40.215231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.275002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a763bc01b4049529949ee8c788e435f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.275672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a763bc01b4049529949ee8c788e435f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.276417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a763bc01b4049529949ee8c788e435f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.285050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a763bc01b4049529949ee8c788e435f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.302105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a763bc01b4049529949ee8c788e435f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.320071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1013/2034] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:40 2022] DELETE /identity/v3/users/1a763bc01b4049529949ee8c788e435f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:40.427187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:40.428731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.442194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a7c5-fa6b-4b57-91b2-737bd9f647a4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:40.507141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:40.507794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.507973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.508429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] PATH_INFO: `/v3/users/1cb5b1eb9c784fd38266ac9851ade54e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.509695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cb5b1eb9c784fd38266ac9851ade54e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:40.511826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.549180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:40.552044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.575761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9c52-11d2-4c33-82c4-761bd0e5bae4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:40.582945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:40.583586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.583875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.584159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] PATH_INFO: `/v3/projects/d1e9922d50f0452ab1a4bee0543f4773` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.585448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e9922d50f0452ab1a4bee0543f4773)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:40.588215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.601438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cb5b1eb9c784fd38266ac9851ade54e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.602010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cb5b1eb9c784fd38266ac9851ade54e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.603882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cb5b1eb9c784fd38266ac9851ade54e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.623602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cb5b1eb9c784fd38266ac9851ade54e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.644215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cb5b1eb9c784fd38266ac9851ade54e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.665740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1022/2035] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:40 2022] DELETE /identity/v3/users/1cb5b1eb9c784fd38266ac9851ade54e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:40.666971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:40.669882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.670911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e9922d50f0452ab1a4bee0543f4773', 'request_id': 'req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.684041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e9922d50f0452ab1a4bee0543f4773', 'request_id': 'req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.690800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e9922d50f0452ab1a4bee0543f4773', 'request_id': 'req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.709079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] The token cache is being invalidate because project d1e9922d50f0452ab1a4bee0543f4773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:40.709371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.710034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e8a688b0-e0d9-4d50-905f-37f73e10d4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.711937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1014/2036] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:40 2022] DELETE /identity/v3/projects/d1e9922d50f0452ab1a4bee0543f4773 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:40.712941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ca03-6c8e-4154-9e69-d65d4f84099d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.713229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ca03-6c8e-4154-9e69-d65d4f84099d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.713502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ca03-6c8e-4154-9e69-d65d4f84099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.732988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:40.733502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d240ca03-6c8e-4154-9e69-d65d4f84099d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:40.734771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d240ca03-6c8e-4154-9e69-d65d4f84099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.758049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:40.760002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2a65e-3444-4740-ac01-bad8e88b0f80 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:40.766545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:40.767208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.767495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.767782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] PATH_INFO: `/v3/projects/d1f5593b589846538753b8919b8da579` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.769081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f5593b589846538753b8919b8da579)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:40.772855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.806859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1015/2037] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:40.807969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5625-78eb-4f83-b5ce-43ff3e463d77 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.808266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5625-78eb-4f83-b5ce-43ff3e463d77 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.808532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5625-78eb-4f83-b5ce-43ff3e463d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.834264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f5593b589846538753b8919b8da579', 'request_id': 'req-c97be1a2-90c7-48ca-af79-9770ca0e2c91'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.846879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-092f5625-78eb-4f83-b5ce-43ff3e463d77 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:40.847544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f5593b589846538753b8919b8da579', 'request_id': 'req-c97be1a2-90c7-48ca-af79-9770ca0e2c91'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.848508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-092f5625-78eb-4f83-b5ce-43ff3e463d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.856592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f5593b589846538753b8919b8da579', 'request_id': 'req-c97be1a2-90c7-48ca-af79-9770ca0e2c91'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.868257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] The token cache is being invalidate because project d1f5593b589846538753b8919b8da579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:40.868550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.869217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:40.872467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1023/2038] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:40 2022] DELETE /identity/v3/projects/d1f5593b589846538753b8919b8da579 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:40.877284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:40.879507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.950481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1016/2039] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:40.951747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d077e-4b41-4b13-bf4f-7b1469c9b0b0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.952031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d077e-4b41-4b13-bf4f-7b1469c9b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.952305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d077e-4b41-4b13-bf4f-7b1469c9b0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.959292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97be1a2-90c7-48ca-af79-9770ca0e2c91 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:40.964551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:40.965412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.965861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.966177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.968309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.977406 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4d0d077e-4b41-4b13-bf4f-7b1469c9b0b0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:40.978778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4d0d077e-4b41-4b13-bf4f-7b1469c9b0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:40.984087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:40.985977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564d1f0-c668-46df-9287-14afdace49fe service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:40.987253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1017/2040] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:40.988330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.988649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:40.988920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:40.997833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1024/2041] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:40.999272 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:40.999800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.000975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.012918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:41.015067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.024991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:41.026459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.027276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1018/2042] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:41.031721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.039012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1025/2043] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:41.043428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158432-00fd-4356-8061-15f7d34f5434 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.044149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.049864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.051560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.052069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.052625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.053602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:41.054141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.055067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38d4d-65c9-492f-a087-29252e7a4720 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.056512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.060238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.061038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.061330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.061969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.063200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:41.063704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.065545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.076323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1019/2044] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:41.082380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.091847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1026/2045] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:41.093071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.097490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cebd6d-12b3-4a82-be63-0f50c339c092 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.105254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.105532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162acf4c-c96f-46cd-8fc2-1116a418fcbc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.106703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.107366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.107846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.108794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.110939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.111566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.111855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.112291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.112956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.113419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.115277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.151085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1020/2046] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:41.155551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.159989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1027/2047] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:41.160697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.219072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:41.244835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b144f-3fdd-4724-a6e2-6271ce08e57e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.262158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eff75a-0391-4bb1-b7aa-e86d2a5cca7b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.271600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.272987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.273546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.274064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.274558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:41.275770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.279055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.310545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1028/2048] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:41.319397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.344726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.345835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.346635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.346924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.347696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.349844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.360352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e1539-9925-455f-819c-9611b1c035ae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.382218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.382950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.383234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.384932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.385597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.387086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1021/2049] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:41.387265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.387869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.392617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.404115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1029/2050] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:41.405013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.420066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3025fd-ed39-488e-90cc-d4f5b29d79c4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.424954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010e988-ffa5-42d7-8432-8352fef6bb9e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.426186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.426839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.427127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.427458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] PATH_INFO: `/v3/projects/923fa2148c934284a04514ceda1b227e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.429157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923fa2148c934284a04514ceda1b227e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.430619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.431241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.431555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.431833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.432484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.432591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.434619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.447145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1030/2051] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:41.447518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.459453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a44e5-9340-49a1-acca-e590e2a35d1c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.464544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.465829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.465829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.466020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.466610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.468268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.482851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1031/2052] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:41.483771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.497371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc5832-deb4-422d-898e-9e1cf048a176 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.498512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923fa2148c934284a04514ceda1b227e', 'request_id': 'req-55d72b38-1727-4ab8-a98a-11f09002be13'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.506080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923fa2148c934284a04514ceda1b227e', 'request_id': 'req-55d72b38-1727-4ab8-a98a-11f09002be13'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.512243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923fa2148c934284a04514ceda1b227e', 'request_id': 'req-55d72b38-1727-4ab8-a98a-11f09002be13'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.531019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] The token cache is being invalidate because project 923fa2148c934284a04514ceda1b227e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:41.532102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.533387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.535267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1022/2053] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:41 2022] DELETE /identity/v3/projects/923fa2148c934284a04514ceda1b227e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:41.535918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.537342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.581424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.582076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.582388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.582670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.583315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.584917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.595575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1032/2054] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:41.596368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.599898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.611005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d72b38-1727-4ab8-a98a-11f09002be13 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.612561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864558b-2447-4591-be47-f5730f0439dc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.615823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.616493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.616777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.617050 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] PATH_INFO: `/v3/projects/a99bc8f4269c4ac89db4b1dee7bf3b31/users/e6b80bea265f4574a40d3b1f5bcd7565/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.617274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.617922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.618209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.618487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] PATH_INFO: `/v3/users/9de8a27c2f224a15ab1a9aa4c7a6da93` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.620269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de8a27c2f224a15ab1a9aa4c7a6da93)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.622233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.634637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99bc8f4269c4ac89db4b1dee7bf3b31, user_id=e6b80bea265f4574a40d3b1f5bcd7565, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.637368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.652222 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1023/2055] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:41 2022] PUT /identity/v3/projects/a99bc8f4269c4ac89db4b1dee7bf3b31/users/e6b80bea265f4574a40d3b1f5bcd7565/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:41.662598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.664316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.677972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de8a27c2f224a15ab1a9aa4c7a6da93'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.679023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de8a27c2f224a15ab1a9aa4c7a6da93'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.679673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de8a27c2f224a15ab1a9aa4c7a6da93'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.689583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de8a27c2f224a15ab1a9aa4c7a6da93'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.689916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:41.710063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e0c070d9-1e49-4228-9a99-2b1c9fdb0acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de8a27c2f224a15ab1a9aa4c7a6da93'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:41.715062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:41.738981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1033/2056] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:41 2022] DELETE /identity/v3/users/9de8a27c2f224a15ab1a9aa4c7a6da93 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:41.740357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.740645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.740918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.751701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bbf0-4486-4f6e-ac74-54817e5178a9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.793015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] MFA Rules not processed for user `e6b80bea265f4574a40d3b1f5bcd7565`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:41.794655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.867432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:41.868143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.868426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.868702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] PATH_INFO: `/v3/projects/0ad40ea9d44046a389c3f28bc6fc11f4/users/5e0307f8e983453289f89b7c0e600c6a/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.908007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1034/2057] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:41.908316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad40ea9d44046a389c3f28bc6fc11f4, user_id=5e0307f8e983453289f89b7c0e600c6a, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:41.912987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142d08d-400d-47ef-b597-4feea6a6a2c0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:41.913193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:41.918156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:41.955069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c2004-7ff2-4d1f-a961-d86b6efa6aff None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:41.956221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1024/2058] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:41 2022] PUT /identity/v3/projects/0ad40ea9d44046a389c3f28bc6fc11f4/users/5e0307f8e983453289f89b7c0e600c6a/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:41.958761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:41.959049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:41.959907 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:41.999731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] MFA Rules not processed for user `5e0307f8e983453289f89b7c0e600c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:42.001133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.038207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.038770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.039059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.039335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.039974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.042335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.084276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1035/2059] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:42.087818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.107701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1025/2060] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:42.112238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.115976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.128091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe61de-02ce-405d-8b8d-46f413bd0f39 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.169690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab0f7-153c-4551-bb60-6024da2c27f9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.175684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:42.176229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.176872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.177124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.177393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.178062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.180900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.200073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:42.202565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.204006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.204296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.204570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.205221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.206518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1036/2061] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:42 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:42.207370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.207499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.210050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.224158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadf75-b1ed-453c-9220-1d0e80bba4bc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.229298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.229993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.230411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.230686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] PATH_INFO: `/v3/projects/37b28854b3524a8e9f77cad66f5a1d88` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.231809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b28854b3524a8e9f77cad66f5a1d88)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.233866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.238234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1026/2062] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:42 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:42.238234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.285914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b28854b3524a8e9f77cad66f5a1d88', 'request_id': 'req-fefcebd5-fa34-4666-a4b1-a302e488ac4f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.301744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b28854b3524a8e9f77cad66f5a1d88', 'request_id': 'req-fefcebd5-fa34-4666-a4b1-a302e488ac4f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.310577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0ceb1-029f-4c37-8c01-e8262f1e1182 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.317834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b28854b3524a8e9f77cad66f5a1d88', 'request_id': 'req-fefcebd5-fa34-4666-a4b1-a302e488ac4f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.331750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] The token cache is being invalidate because project 37b28854b3524a8e9f77cad66f5a1d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:42.332041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.332743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.334298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1037/2063] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:42 2022] DELETE /identity/v3/projects/37b28854b3524a8e9f77cad66f5a1d88 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:42.335153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.336529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.395751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.396363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.396923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.397202 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.397895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.399562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.425854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1027/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:42.427180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.431822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.434988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcebd5-fa34-4666-a4b1-a302e488ac4f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.446597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ab5d7-ae78-42a8-84f5-3da4521b2498 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.448329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.449179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.449770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.450237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.451335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.455115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.455394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.456084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.456373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.456640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] PATH_INFO: `/v3/users/55ce7c30cfb84f20b469f06abfbaa5cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.458226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ce7c30cfb84f20b469f06abfbaa5cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.460188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.480223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1038/2065] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:42 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:42.481096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.484172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.499555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d65a7-044f-446e-a92c-e05e61250949 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.505072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.505845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.506130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.506566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] PATH_INFO: `/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05/users/6f88de2cb910405888a106bdb05a0995/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.519325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ce7c30cfb84f20b469f06abfbaa5cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.519994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ce7c30cfb84f20b469f06abfbaa5cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.520622 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ce7c30cfb84f20b469f06abfbaa5cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.521996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199139dd4cad431e9e7f1b0d0bcecc05, user_id=6f88de2cb910405888a106bdb05a0995, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.524677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.527510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ce7c30cfb84f20b469f06abfbaa5cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.534052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:42.539433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1039/2066] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:42 2022] PUT /identity/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05/users/6f88de2cb910405888a106bdb05a0995/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:42.540446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.542857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.543993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ce7c30cfb84f20b469f06abfbaa5cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:42.560892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1028/2067] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:42 2022] DELETE /identity/v3/users/55ce7c30cfb84f20b469f06abfbaa5cb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:42.560892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.560892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.611338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:42.633300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae757-a993-4536-93b0-bd4d041074f7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.640115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.640818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.641110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.641390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.642122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.642248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b33f9-6371-4b65-9e4c-78c38d04e440 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.643986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.648227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.648390 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.648810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.649092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.650143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.651469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.659100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1040/2068] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:42.660129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.662974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1029/2069] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:42.668142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.672742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4267d-1e82-4c43-ab4c-db2608d2caca admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.679391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.680103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.680408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.680770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] PATH_INFO: `/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f/users/6621ede73dc64e519c361e716b3925cf/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.685245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d7e9-4588-4ddb-a254-31c12fd6acbd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.692890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.693826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.693952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.694168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] PATH_INFO: `/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05/users/6f88de2cb910405888a106bdb05a0995/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.696385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199139dd4cad431e9e7f1b0d0bcecc05, user_id=6f88de2cb910405888a106bdb05a0995, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.698812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a950e7b089641c6b1f8d046e2d41d1f, user_id=6621ede73dc64e519c361e716b3925cf, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.699081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.701473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487c65b-c336-417d-afa2-c2953fd74d8c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.723093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1030/2070] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:42 2022] PUT /identity/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05/users/6f88de2cb910405888a106bdb05a0995/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:42.723323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1041/2071] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:42 2022] PUT /identity/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f/users/6621ede73dc64e519c361e716b3925cf/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:42.723415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.724762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.724876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.725088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.767058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] MFA Rules not processed for user `6f88de2cb910405888a106bdb05a0995`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:42.768297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.798179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d727c-5b13-48f9-a0e0-ea8b00ca21c3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.853894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1042/2072] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:42.865613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.867221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.882120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.886086 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.886380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.886803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.887336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.889447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.899855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1031/2073] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:42.900586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.902514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:42.917045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b634d0c-da36-4a99-9819-a3e20d946954 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.922552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.923203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.923515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.923788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] PATH_INFO: `/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.924978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e9893d64a44d93bb79ae591a84ed4a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.927001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.928560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:42.933506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982480ed-3a4c-4373-a511-444e58ed134a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.938671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:42.939283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:42.939588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:42.939956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:42.951683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:42.953185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:42.965188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1043/2074] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:42.966085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:42.979140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6f95-a70a-4f0e-9288-665d71eb064b service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:42.996117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e9893d64a44d93bb79ae591a84ed4a', 'request_id': 'req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:43.006350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e9893d64a44d93bb79ae591a84ed4a', 'request_id': 'req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:43.014463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e9893d64a44d93bb79ae591a84ed4a', 'request_id': 'req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:43.038004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] The token cache is being invalidate because project e3e9893d64a44d93bb79ae591a84ed4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:43.038377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:43.038989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54b4f72-bcb0-4cce-bb10-56cc1fe9eed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:43.040542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1032/2075] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:42 2022] DELETE /identity/v3/projects/e3e9893d64a44d93bb79ae591a84ed4a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:43.042356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:43.046536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704ffd0-d6c3-4197-b452-6b0630fb768f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.046936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704ffd0-d6c3-4197-b452-6b0630fb768f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.047869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704ffd0-d6c3-4197-b452-6b0630fb768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.059072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.059801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.060426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.060760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] PATH_INFO: `/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f/users/6621ede73dc64e519c361e716b3925cf/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.063486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a950e7b089641c6b1f8d046e2d41d1f, user_id=6621ede73dc64e519c361e716b3925cf, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.066319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1edd9-7f58-4c2c-a200-f42f8265d505 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.087641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1044/2076] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:42 2022] PUT /identity/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f/users/6621ede73dc64e519c361e716b3925cf/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:43.087930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8704ffd0-d6c3-4197-b452-6b0630fb768f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:43.089376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8704ffd0-d6c3-4197-b452-6b0630fb768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.099652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.100239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.100858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.139220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] MFA Rules not processed for user `6621ede73dc64e519c361e716b3925cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:43.140556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.194374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1033/2077] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:43.203322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.204393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.204678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.228573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:43.229989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.240725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1034/2078] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:43.241181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1045/2079] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:43.245087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.252651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.254207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.257383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d7015-64cf-4d58-8eef-89f62271bf67 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.262215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.262825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.263156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.263538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.264193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:43.264500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.266733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.285767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1046/2080] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:43.286779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.298710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276e0e6-3838-489d-b5e4-53cf54a0d273 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.304273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.305954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.306256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.306400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.307141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.309774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.321843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edb5f9-5b73-4599-8518-a5ac04abfb8d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.326503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.327141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.327501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.327860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.330359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.352119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1047/2081] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/projects => generated 397 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:43.353068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.401625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:43.434010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.434010 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.442977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1035/2082] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:43.454755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eac621-6f5b-43c8-a7a5-5e1ca985107c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.460144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.460738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.461160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.461442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.462255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.464200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.490036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1048/2083] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:43.494292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.506159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5818f9c-0238-4407-bf39-ba659f1e7073 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.510804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.511409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.511689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.511961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.512603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.514236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.524527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1036/2084] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:43.528677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.538940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db41c73-6963-414b-acd4-4890495dee46 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.547770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.548741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.550348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.551362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.553189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.558820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3bb9-1537-4e98-86aa-e7fb9a5b84f4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.573909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1049/2085] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:43.580507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.591573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07d10-ec36-4e66-b244-7a4e58e80535 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.596532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.597213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.597496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.597807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] PATH_INFO: `/v3/projects/656e9c75b8cf41d5a4cbcb7390eda927/users/7d08548f6ecf4a389ecc23f5a1868e16/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.622324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=656e9c75b8cf41d5a4cbcb7390eda927, user_id=7d08548f6ecf4a389ecc23f5a1868e16, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.625909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.640756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1037/2086] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:43 2022] PUT /identity/v3/projects/656e9c75b8cf41d5a4cbcb7390eda927/users/7d08548f6ecf4a389ecc23f5a1868e16/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:43.652339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.652921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.653770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.679463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:43.691829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] MFA Rules not processed for user `7d08548f6ecf4a389ecc23f5a1868e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:43.697640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:43.808613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1050/2087] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:43.824740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:43.916632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06afe12-c014-4327-a8ee-bafe0c91f65f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:43.922820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:43.923498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:43.924016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:43.924155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:43.937247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:43.939119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7486a14-e809-4276-9400-ca8eb7404f0b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:43.950571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1038/2088] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:44.665578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:44.666997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:44.732851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5881f1-9560-4e1e-b7fa-71899bef7694 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:44.737498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:44.738190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:44.738506 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:44.738782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] PATH_INFO: `/v3/users/e6b80bea265f4574a40d3b1f5bcd7565` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:44.739938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b80bea265f4574a40d3b1f5bcd7565)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:44.742047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:44.786941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b80bea265f4574a40d3b1f5bcd7565'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:44.787593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b80bea265f4574a40d3b1f5bcd7565'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:44.788201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b80bea265f4574a40d3b1f5bcd7565'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:44.795264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b80bea265f4574a40d3b1f5bcd7565'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:44.808035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b80bea265f4574a40d3b1f5bcd7565'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:44.824762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1051/2089] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:44 2022] DELETE /identity/v3/users/e6b80bea265f4574a40d3b1f5bcd7565 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:44.829875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:44.830355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:44.830484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:44.851167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:44.852527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:44.913361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1039/2090] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:44.928828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:44.930546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:44.954593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:44.997376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ae20a-0e24-4fc2-a214-f0e91b2147d4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:45.003274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:45.003887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:45.004167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:45.004440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:45.013601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:45.015059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:45.024841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1052/2091] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:45.308419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:45.310282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:45.328037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778098e-01b7-40b5-a95c-3ec2a67ae245 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:45.345306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:45.346073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:45.346359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:45.346713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] PATH_INFO: `/v3/projects/a99bc8f4269c4ac89db4b1dee7bf3b31` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:45.348060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99bc8f4269c4ac89db4b1dee7bf3b31)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:45.349965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:45.394932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:45.402217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99bc8f4269c4ac89db4b1dee7bf3b31', 'request_id': 'req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.409675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99bc8f4269c4ac89db4b1dee7bf3b31', 'request_id': 'req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.415383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99bc8f4269c4ac89db4b1dee7bf3b31', 'request_id': 'req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.428467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] The token cache is being invalidate because project a99bc8f4269c4ac89db4b1dee7bf3b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:45.428761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.429416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.430898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1040/2092] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:45 2022] DELETE /identity/v3/projects/a99bc8f4269c4ac89db4b1dee7bf3b31 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:45.434964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:45.444842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c485fb2-a7de-445f-9b24-7ec3c1f5e514 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:45.507356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:45.508080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:45.508436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:45.508707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] PATH_INFO: `/v3/users/6f88de2cb910405888a106bdb05a0995` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:45.510557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f88de2cb910405888a106bdb05a0995)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:45.512631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:45.567719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f88de2cb910405888a106bdb05a0995'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.568378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f88de2cb910405888a106bdb05a0995'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.569038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f88de2cb910405888a106bdb05a0995'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.582587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f88de2cb910405888a106bdb05a0995'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.598764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f88de2cb910405888a106bdb05a0995'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:45.621771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1053/2093] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:45 2022] DELETE /identity/v3/users/6f88de2cb910405888a106bdb05a0995 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:46.034779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.036982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.053062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ed328-6591-4097-9fd5-9d5aa7cc7a8e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.087805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.092127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.136143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.136875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.137178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.137452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] PATH_INFO: `/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.138688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199139dd4cad431e9e7f1b0d0bcecc05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.141283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.146384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:46.179129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3b571-6980-4037-a59f-58f345fad896 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.190660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.191317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.191599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.191922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] PATH_INFO: `/v3/users/5e0307f8e983453289f89b7c0e600c6a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.193388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0307f8e983453289f89b7c0e600c6a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.195643 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.215652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199139dd4cad431e9e7f1b0d0bcecc05', 'request_id': 'req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.226312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199139dd4cad431e9e7f1b0d0bcecc05', 'request_id': 'req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.242379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199139dd4cad431e9e7f1b0d0bcecc05', 'request_id': 'req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.267324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0307f8e983453289f89b7c0e600c6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.267539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] The token cache is being invalidate because project 199139dd4cad431e9e7f1b0d0bcecc05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:46.267763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.268258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0307f8e983453289f89b7c0e600c6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.269154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0307f8e983453289f89b7c0e600c6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.269267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a8dd2fe4-45cb-4866-9d4a-67354e3cf479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.271021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1041/2094] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:46 2022] DELETE /identity/v3/projects/199139dd4cad431e9e7f1b0d0bcecc05 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:46.275804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0aeb-94c0-46f1-a884-576b045a55a8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.276134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0aeb-94c0-46f1-a884-576b045a55a8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.276365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0aeb-94c0-46f1-a884-576b045a55a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.280176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0307f8e983453289f89b7c0e600c6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.293335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7caca657-74fb-4634-affd-4e875e6742b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0307f8e983453289f89b7c0e600c6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.296196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9b0d0aeb-94c0-46f1-a884-576b045a55a8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:46.297450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b0d0aeb-94c0-46f1-a884-576b045a55a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.307854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1054/2095] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:46 2022] DELETE /identity/v3/users/5e0307f8e983453289f89b7c0e600c6a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:46.312473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.313959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.314244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.334430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:46.335659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.351215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:46.362318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1042/2096] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:46.369913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.370278 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.370649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.391444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:46.392643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.406131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1055/2097] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:46.415690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.417143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.451867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1043/2098] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:46.456426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.465229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c4fc3-aae4-4638-bb6f-9562e8279c2c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.469875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.470487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.470770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.471041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.471682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:46.471983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.473580 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.475388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09db326-4efc-45ef-829f-90f46093c3eb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.479650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.480254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.480540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.480821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.488759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1044/2099] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:46.489667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.490960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.492394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.498997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba050-3474-4b52-a8ad-e9b64513778d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.502631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1056/2100] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:46.504398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.504981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.505260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.505535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.506189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.508162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.539717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1045/2101] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:46.543786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.550996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:46.554939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911f039-438c-4b3e-b2d7-06ff88b2daa5 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.618607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.619389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.619755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.620107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.620828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.622918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.648828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1057/2102] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:46.650179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.664082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e2b79-6c49-408b-af8c-d182c1d113fc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.669642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.670294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.670581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.670863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.671518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.673678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.684214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1046/2103] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:46.688738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.704224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aea1-683b-4af4-9ee5-9b1d34dbe400 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.709417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.710096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.710321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.710781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.711437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.713204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.726616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1058/2104] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:46.732507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.745168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004b167-ae1b-456d-97a3-7c2c229d489f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.751253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.752178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.752531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.752877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] PATH_INFO: `/v3/projects/8882040969cc4fef96466733d9904102/users/47b852f73ebe4046af3831ef0ff8ff50/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.773218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8882040969cc4fef96466733d9904102, user_id=47b852f73ebe4046af3831ef0ff8ff50, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.776769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a6f4c-ea6d-4844-81d5-bf6958b93fd4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.805739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1047/2105] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:46 2022] PUT /identity/v3/projects/8882040969cc4fef96466733d9904102/users/47b852f73ebe4046af3831ef0ff8ff50/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:46.810886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.813051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.816796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.817124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.817376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.832548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95f519-ee01-4eeb-85a9-66b63a2fd22b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.861201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] MFA Rules not processed for user `47b852f73ebe4046af3831ef0ff8ff50`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:46.862881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.893392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:46.894063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:46.894349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:46.894628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] PATH_INFO: `/v3/projects/0ad40ea9d44046a389c3f28bc6fc11f4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:46.895759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad40ea9d44046a389c3f28bc6fc11f4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:46.897679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:46.929220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1048/2106] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:46.933929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.945235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b24e-adfc-4d7d-8566-82e39c116cf0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:46.946930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad40ea9d44046a389c3f28bc6fc11f4', 'request_id': 'req-056c14fb-a794-4fab-82e7-18a81fedddf1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.955912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad40ea9d44046a389c3f28bc6fc11f4', 'request_id': 'req-056c14fb-a794-4fab-82e7-18a81fedddf1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.963684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad40ea9d44046a389c3f28bc6fc11f4', 'request_id': 'req-056c14fb-a794-4fab-82e7-18a81fedddf1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.964396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:46.976432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] The token cache is being invalidate because project 0ad40ea9d44046a389c3f28bc6fc11f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:46.976726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.977391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:46.978952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1059/2107] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:46 2022] DELETE /identity/v3/projects/0ad40ea9d44046a389c3f28bc6fc11f4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:46.982741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:46.984147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:46.997750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:47.011405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.012027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.012313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.012590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.013232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.015486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.047611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1049/2108] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:46 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:47.049315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.050947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.054883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c14fb-a794-4fab-82e7-18a81fedddf1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.114288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.115066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.115433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.115710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] PATH_INFO: `/v3/users/6621ede73dc64e519c361e716b3925cf` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.116860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6621ede73dc64e519c361e716b3925cf)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.118844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.118951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921a97-98ed-4c12-a8df-df6cb4958e88 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.123609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.124203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.124484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.124756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.125955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.127863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.150250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1050/2109] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:47.154403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.164141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8828-5a30-4bbb-b690-e7872e85ebb2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.167955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6621ede73dc64e519c361e716b3925cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.168623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6621ede73dc64e519c361e716b3925cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.168774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.169257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6621ede73dc64e519c361e716b3925cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.169361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.169621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.169933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.170656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.172283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.176285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6621ede73dc64e519c361e716b3925cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.182048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1051/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:47.185989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.188233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.189162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6621ede73dc64e519c361e716b3925cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.200863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aede6b-fe64-4489-9e2b-7984b02b8f1c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.202396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1060/2111] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:46 2022] DELETE /identity/v3/users/6621ede73dc64e519c361e716b3925cf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:47.264316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.265002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.265292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.265624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] PATH_INFO: `/v3/projects/ee9d2de897b34d04a360b5a52905bd08/users/af80cca591a14bb588a975ebe49b141b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.285254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9d2de897b34d04a360b5a52905bd08, user_id=af80cca591a14bb588a975ebe49b141b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.288200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.304011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1052/2112] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:47 2022] PUT /identity/v3/projects/ee9d2de897b34d04a360b5a52905bd08/users/af80cca591a14bb588a975ebe49b141b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:47.312156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.319922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503444df-88ac-4b48-809c-dd475ec4cef0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.363453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:47.399050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.399782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.400130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.400474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.401192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.402896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73047-6593-4eba-91bf-92cea035f837 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.413330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1061/2113] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:47.417744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.428892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c007-9e3e-4096-b232-93e061ca882c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.433694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.434790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.435141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.435536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] PATH_INFO: `/v3/projects/ee9d2de897b34d04a360b5a52905bd08/users/af80cca591a14bb588a975ebe49b141b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.438205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9d2de897b34d04a360b5a52905bd08, user_id=af80cca591a14bb588a975ebe49b141b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.440862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.443693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:47.454080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1053/2114] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:47 2022] PUT /identity/v3/projects/ee9d2de897b34d04a360b5a52905bd08/users/af80cca591a14bb588a975ebe49b141b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:47.464590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.464896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.465235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.491588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.493343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.497043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] MFA Rules not processed for user `af80cca591a14bb588a975ebe49b141b`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:47.499230 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.571357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1062/2115] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:47.574482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb435c14-668e-41bb-a893-9be1dd6bbfa0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.579754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.580423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.580708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.580983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] PATH_INFO: `/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.582172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a950e7b089641c6b1f8d046e2d41d1f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.582890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.584294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.584408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.632791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a950e7b089641c6b1f8d046e2d41d1f', 'request_id': 'req-7f47ed11-3d3c-4973-b1fa-3bec393226ca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.639509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a950e7b089641c6b1f8d046e2d41d1f', 'request_id': 'req-7f47ed11-3d3c-4973-b1fa-3bec393226ca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.642349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00155d11-84f8-402d-92fa-b0786d4eefec None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.646240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a950e7b089641c6b1f8d046e2d41d1f', 'request_id': 'req-7f47ed11-3d3c-4973-b1fa-3bec393226ca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.659473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] The token cache is being invalidate because project 6a950e7b089641c6b1f8d046e2d41d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:47.659767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.660429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7f47ed11-3d3c-4973-b1fa-3bec393226ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.662015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1054/2116] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:47 2022] DELETE /identity/v3/projects/6a950e7b089641c6b1f8d046e2d41d1f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:47.666617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715740-50c0-472e-aa90-eff3a007f108 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.666904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715740-50c0-472e-aa90-eff3a007f108 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.667191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715740-50c0-472e-aa90-eff3a007f108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.686726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cb715740-50c0-472e-aa90-eff3a007f108 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:47.687995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb715740-50c0-472e-aa90-eff3a007f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.691748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.692357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.692638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.692914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.694703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.726699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:47.750136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1055/2117] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:47.757567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.757886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.758162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.764297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.766522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.777366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1063/2118] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:47.780491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:47.782228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.790494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1056/2119] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:47.794710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.814867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62369d-7c68-4ad7-b0f7-a968c79f600e service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.823300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.823924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.824189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.824481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.825129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:47.825439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.827205 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.843147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1064/2120] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:47.846188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.854359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.856142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:47.856270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013ff3-bfc3-456b-98dd-756376e4b352 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.861830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.862437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.862724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.863002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.863638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.865551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.869241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdf0ad-6649-4245-add1-a33f66ef1fee admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.873870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:47.874483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.874761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.875055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] PATH_INFO: `/v3/users/47b852f73ebe4046af3831ef0ff8ff50` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.876200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b852f73ebe4046af3831ef0ff8ff50)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:47.878203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:47.885458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:47.901859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1057/2121] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:47.902726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:47.928587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b852f73ebe4046af3831ef0ff8ff50'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.929292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b852f73ebe4046af3831ef0ff8ff50'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.929960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b852f73ebe4046af3831ef0ff8ff50'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.937151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b852f73ebe4046af3831ef0ff8ff50'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.949642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-17357285-da65-4209-a1b4-c5aa1f2f9621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b852f73ebe4046af3831ef0ff8ff50'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:47.963468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1065/2122] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:47 2022] DELETE /identity/v3/users/47b852f73ebe4046af3831ef0ff8ff50 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:47.967566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:47.967855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:47.968134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:47.968592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf3682-17b2-4697-a6c8-94fbe0e941a3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:47.987937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:47.989055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.030970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.033051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.033769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.034064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.034713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.036699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.051036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1066/2123] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:48.061227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1058/2124] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:47 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:48.061685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.062739 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.062922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.064475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.077986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c81c31-6b1d-4ba7-9bf5-0373c0f324e7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.085243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.085887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.086178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.086457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.087385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.089046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.096245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:48.100948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1059/2125] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:48.104979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.114990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298d8c-c15f-431f-9b1d-5087e05f16b4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.119635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.120217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.120497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.120770 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.121402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.123061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.126723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984e620-e11b-4e63-8784-a839975993a2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.130508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.131105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.131422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.131695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.134830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1060/2126] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:48.138785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.144749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.146208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b8b0-5bc7-4615-8c6d-87b41c5cb8df service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.148678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a9ba-9a1a-423c-ba21-457b291c1e00 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.153275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.153964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.154248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.154522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] PATH_INFO: `/v3/projects/5b0a4831f37e4e199d943c3f2474581d/users/2fab656311164029ac728745e02d80af/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.156061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1067/2127] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:38:48.169558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0a4831f37e4e199d943c3f2474581d, user_id=2fab656311164029ac728745e02d80af, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.172300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.185267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1061/2128] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:48 2022] PUT /identity/v3/projects/5b0a4831f37e4e199d943c3f2474581d/users/2fab656311164029ac728745e02d80af/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:38:48.195718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.196343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.196658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.226339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] MFA Rules not processed for user `2fab656311164029ac728745e02d80af`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:48.228085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.300950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1068/2129] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:48.317020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.318859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.392772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:48.398920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3197c-2683-4bd7-8fcc-f8472860b2b4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.403127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.403747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.404039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.404316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.412837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.414448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.423909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1062/2130] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:48.444434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.446379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:48.516916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae657f-03f4-4a72-9c71-05e723d63e7e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.521361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.522014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.522291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.523196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] PATH_INFO: `/v3/projects/8882040969cc4fef96466733d9904102` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.523680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8882040969cc4fef96466733d9904102)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.525571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.575880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8882040969cc4fef96466733d9904102', 'request_id': 'req-3b61309b-c65a-4094-b374-a400f1a056e1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.584398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8882040969cc4fef96466733d9904102', 'request_id': 'req-3b61309b-c65a-4094-b374-a400f1a056e1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.591614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8882040969cc4fef96466733d9904102', 'request_id': 'req-3b61309b-c65a-4094-b374-a400f1a056e1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.593002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:48.606798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] The token cache is being invalidate because project 8882040969cc4fef96466733d9904102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:48.607091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.607765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.609329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1069/2131] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:48 2022] DELETE /identity/v3/projects/8882040969cc4fef96466733d9904102 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:48.613726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:48.624488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b874d-656c-4397-8de6-67359786ed3e service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:48.696269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:48.697109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:48.697516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:48.697925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] PATH_INFO: `/v3/users/af80cca591a14bb588a975ebe49b141b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:48.699218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=af80cca591a14bb588a975ebe49b141b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:48.701346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:48.759870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af80cca591a14bb588a975ebe49b141b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.760463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af80cca591a14bb588a975ebe49b141b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.761082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af80cca591a14bb588a975ebe49b141b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.770931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af80cca591a14bb588a975ebe49b141b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.783760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cdb09a37-d861-48cd-b544-2f178ce65bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af80cca591a14bb588a975ebe49b141b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:48.796759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1063/2132] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:48 2022] DELETE /identity/v3/users/af80cca591a14bb588a975ebe49b141b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:49.071473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.072992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:49.086496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61309b-c65a-4094-b374-a400f1a056e1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.148275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.148951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.149162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.149431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] PATH_INFO: `/v3/projects/ee9d2de897b34d04a360b5a52905bd08` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.150565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9d2de897b34d04a360b5a52905bd08)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.152449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.199157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9d2de897b34d04a360b5a52905bd08', 'request_id': 'req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:49.211261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9d2de897b34d04a360b5a52905bd08', 'request_id': 'req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:49.218325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9d2de897b34d04a360b5a52905bd08', 'request_id': 'req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:49.229521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] The token cache is being invalidate because project ee9d2de897b34d04a360b5a52905bd08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:49.229844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:49.230517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f1053c0f-edc4-41b4-af6f-fdbca8811cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:49.232094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1070/2133] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:49 2022] DELETE /identity/v3/projects/ee9d2de897b34d04a360b5a52905bd08 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:49.237789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.238263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.238380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.257808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:49.259246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:49.318385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1064/2134] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:49.326131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.326425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.326703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.347034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:49.348347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:49.356466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1071/2135] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:49.361451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.377428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb77bd-1a19-444d-8814-998c498d7eaa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.382191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.382799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.383099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.383629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.384283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:49.384590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.386221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.388836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:49.401351 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1065/2136] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:38:49.405669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.417003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0af3f-ff41-41da-a0ae-04bba801aa99 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.422686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.423351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.423633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.423907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.424545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.426832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.465485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1072/2137] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:49.466459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.478463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3b0a-21a8-412a-b40b-cc2d9373dc86 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.536312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.536923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.537205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.537501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.538146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.540059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.563013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1066/2138] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:49.570490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.582836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a96059-d80f-4b25-b3ee-129e64fe2969 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.589584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.591320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.592400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.592853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.593772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.603109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.618853 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1073/2139] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:49.620442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.631627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df56ec-3404-44ff-9e54-3e5c98fe963b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.637605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.638376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.638667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.638946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.639615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.641863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97877d4-8ff4-4d7d-a369-ad006a70a115 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.656247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1067/2140] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:49.660969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.672101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6ff6b-a9d5-472b-bb0b-b98f6c7f092f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:49.677404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:49.678441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.678737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.679021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] PATH_INFO: `/v3/projects/214054d2485e495d8730065b38201cbc/users/f66e746556904d94a25236f6b9476d6a/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.706836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=214054d2485e495d8730065b38201cbc, user_id=f66e746556904d94a25236f6b9476d6a, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:49.712128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:49.735605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1074/2141] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:49 2022] PUT /identity/v3/projects/214054d2485e495d8730065b38201cbc/users/f66e746556904d94a25236f6b9476d6a/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:49.743110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:49.743546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:49.743940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:49.789324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] MFA Rules not processed for user `f66e746556904d94a25236f6b9476d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:49.791229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:49.892722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1068/2142] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:49.894077 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:49.923101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:50.011456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ed2c2-c50a-4e24-8f42-23fe66c3cd33 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.018604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.019409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.019847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.020120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.020768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.023757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.072535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1075/2143] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:49 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:50.076696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.095123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d803d5f-ebfc-4e19-ae21-6054a21e9f3f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.175563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.176677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.176960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.177260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.180601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.180601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.192427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:50.212382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1069/2144] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:50 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:50.212490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.222333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ffe13-6e47-4fd5-b269-dd3b0ae58fdc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.226978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.227713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.228037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.228281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.228929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.230578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.242495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1076/2145] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:38:50.246739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.258359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cf935-29aa-4a86-8be9-c5ab840466db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.263421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.264117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.264364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.264638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] PATH_INFO: `/v3/projects/8402340029bf45e5a5ed4f754683f836/users/bb409787eac84096a5ea0ec44c45fb57/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.282764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8402340029bf45e5a5ed4f754683f836, user_id=bb409787eac84096a5ea0ec44c45fb57, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.285729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.300936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1070/2146] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:50 2022] PUT /identity/v3/projects/8402340029bf45e5a5ed4f754683f836/users/bb409787eac84096a5ea0ec44c45fb57/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:50.304824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.315493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f409b-20a4-4140-91e8-046990c7f147 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.377155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.377804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.378088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.378360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.379031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.380655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0d326-1308-4a6e-b817-39feb71ce34a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.390685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1077/2147] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:50.395067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.405218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc291-8faf-462e-911e-3146ef0f6278 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.410261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.410921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.411206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.411484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] PATH_INFO: `/v3/projects/8402340029bf45e5a5ed4f754683f836/users/bb409787eac84096a5ea0ec44c45fb57/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.413449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8402340029bf45e5a5ed4f754683f836, user_id=bb409787eac84096a5ea0ec44c45fb57, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.416121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.427966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1071/2148] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:50 2022] PUT /identity/v3/projects/8402340029bf45e5a5ed4f754683f836/users/bb409787eac84096a5ea0ec44c45fb57/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:50.438932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.439251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.439533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.467257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] MFA Rules not processed for user `bb409787eac84096a5ea0ec44c45fb57`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:50.468472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:50.539277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1078/2149] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:50.550740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:50.552420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:50.611889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305706d-7f9e-4e06-be5c-b7f8521a86c5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:50.615527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:50.616129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:50.616410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:50.616684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:50.625590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:50.626993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359470-7f51-48aa-b94b-8ee364c79574 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:50.636204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1072/2150] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:51.053237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:51.070525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:51.134192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145640b-d78b-4595-853d-830ac7fd3f5a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:51.138831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:51.139502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:51.139783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:51.140537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] PATH_INFO: `/v3/users/f66e746556904d94a25236f6b9476d6a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:51.141401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66e746556904d94a25236f6b9476d6a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:51.143388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:51.190567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66e746556904d94a25236f6b9476d6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.191159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66e746556904d94a25236f6b9476d6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.191798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66e746556904d94a25236f6b9476d6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.199223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66e746556904d94a25236f6b9476d6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.210992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66e746556904d94a25236f6b9476d6a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.225262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1079/2151] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:51 2022] DELETE /identity/v3/users/f66e746556904d94a25236f6b9476d6a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:51.229682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:51.230074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:51.230362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:51.249729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:51.251542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:51.311052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1073/2152] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:51.320803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:51.322515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:51.387246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3877-dca5-4bed-a9fc-b8dc6ba65291 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:51.391557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:51.392158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:51.392440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:51.392712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:51.401559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:51.403021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:51.414966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1080/2153] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:51.690474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:51.692738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:51.707015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e028cd-15ae-4fec-a9d1-5e48d6766097 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:51.711794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:51.712436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:51.712718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:51.712989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] PATH_INFO: `/v3/projects/214054d2485e495d8730065b38201cbc` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:51.714363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=214054d2485e495d8730065b38201cbc)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:51.716428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:51.772480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '214054d2485e495d8730065b38201cbc', 'request_id': 'req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.778900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '214054d2485e495d8730065b38201cbc', 'request_id': 'req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.780186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:51.787592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '214054d2485e495d8730065b38201cbc', 'request_id': 'req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.801411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] The token cache is being invalidate because project 214054d2485e495d8730065b38201cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:51.801876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.802658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.807671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1074/2154] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:51 2022] DELETE /identity/v3/projects/214054d2485e495d8730065b38201cbc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:51.809080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:51.819320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a97d87-5d24-485b-9e41-40c8c0bb540f service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:51.850060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:51.880157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:51.881007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:51.881291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:51.881656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] PATH_INFO: `/v3/users/bb409787eac84096a5ea0ec44c45fb57` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:51.882925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb409787eac84096a5ea0ec44c45fb57)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:51.885101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:51.931436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb409787eac84096a5ea0ec44c45fb57'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.932074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb409787eac84096a5ea0ec44c45fb57'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.932668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb409787eac84096a5ea0ec44c45fb57'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.939810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb409787eac84096a5ea0ec44c45fb57'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.951619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bcacce3-e88b-4e1e-a5d6-c577e52ad8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb409787eac84096a5ea0ec44c45fb57'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:51.964761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1081/2155] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:51 2022] DELETE /identity/v3/users/bb409787eac84096a5ea0ec44c45fb57 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:52.296938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.298921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:52.311718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a6e8b-34b9-4a1f-b8fc-58660814b2a2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.393343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.394023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.394311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.394593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] PATH_INFO: `/v3/projects/8402340029bf45e5a5ed4f754683f836` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.396081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8402340029bf45e5a5ed4f754683f836)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.398077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.448134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8402340029bf45e5a5ed4f754683f836', 'request_id': 'req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:52.455071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8402340029bf45e5a5ed4f754683f836', 'request_id': 'req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:52.461675 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8402340029bf45e5a5ed4f754683f836', 'request_id': 'req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:52.475493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] The token cache is being invalidate because project 8402340029bf45e5a5ed4f754683f836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:52.475792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:52.476438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-111dc93a-0a58-4b55-b0b7-2c33dcd296b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:52.478017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1075/2156] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:52 2022] DELETE /identity/v3/projects/8402340029bf45e5a5ed4f754683f836 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:52.482952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.483437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.483554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.504194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:52.505317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:52.564820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1082/2157] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:52.572970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.573264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.573546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.593354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:52.594609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:52.602489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1076/2158] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:52.606557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.616773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb469964-ca93-4a67-b507-be90bf236495 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.621365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.621996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.622282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.622559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.623200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:38:52.623507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.625753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.637735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1083/2159] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:38:52.643447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.653318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba929b3-8fc3-4b87-90ba-39411831f099 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.658137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.658800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.659068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.659374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.660012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.661954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.690150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1077/2160] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:38:52.694325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.704307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17941ebf-d102-40ed-bc56-83a0a21f62a2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.741993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:52.766793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.767321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.767607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.767887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.768517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.770515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.792032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1084/2161] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:52 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:52.796104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.806379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b97b0-f235-48d8-84b9-32591a7f23be admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.810976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.811569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.811852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.812129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.812812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.814462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.825337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1078/2162] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:52.830006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.840874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e47b7-4b55-4738-9be3-ff1bfc0c42bf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.846533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.847152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.847440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.847721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.848371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.850019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33030b8-92fd-4eb0-b0d8-ffd1a5a991e4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.860623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1085/2163] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:52.864837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:52.875784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a27fa-ff95-4bfe-a2c9-cb0c0d2bd4ae admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:52.880602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:52.881257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.881545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.881878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] PATH_INFO: `/v3/projects/bec0247c2ba948d8aaf054451db1d978/users/2de4dd7826144afdb2d7346e416e804f/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.897028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec0247c2ba948d8aaf054451db1d978, user_id=2de4dd7826144afdb2d7346e416e804f, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:52.899732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:52.912014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1079/2164] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:52 2022] PUT /identity/v3/projects/bec0247c2ba948d8aaf054451db1d978/users/2de4dd7826144afdb2d7346e416e804f/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:52.922106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:52.922415 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:52.922727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:52.954461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] MFA Rules not processed for user `2de4dd7826144afdb2d7346e416e804f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:52.956204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:53.022062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1086/2165] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:53.026589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.036992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:53.095629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cb12a-e8c1-4901-9238-acf2c6205051 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.100956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.101794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.102216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.102498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.103328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.105606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.142387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1080/2166] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:38:53 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:53.146383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.157660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359afe4-a808-4c6a-91f3-904fb9610729 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.215635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.216253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.216538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.216816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.217441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.219930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.241130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1087/2167] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:38:53 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:38:53.245165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.255651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1a7-66c4-4080-b019-f82b8de20359 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.260539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.261137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.261424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.261726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.262378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.263996 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.276006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1081/2168] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:53.280193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.290931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba2ff2-7e53-49f9-a93c-3f17852bc647 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.295576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.296227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.296629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.296814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] PATH_INFO: `/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.312088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e0cd62d91f4863a8e06ba12b6d7d7a, user_id=aa0432cd2b5b4d8f8cd2b1ee44e27d7b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.315456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.331810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1088/2169] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:53 2022] PUT /identity/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:53.336212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.346837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282cce3-d378-4f66-b4a9-ed54070e139d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.405076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.405849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.406146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.409839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.409839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.409839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a4385-e1b8-4ebe-b31d-1a89a86ffff0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.420499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1082/2170] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:38:53.425210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.435605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92620b2b-c182-4eef-a4f9-0a72318ce74b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.441106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.441796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.442082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.442356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] PATH_INFO: `/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.444301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e0cd62d91f4863a8e06ba12b6d7d7a, user_id=aa0432cd2b5b4d8f8cd2b1ee44e27d7b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.447297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.458337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:53.465480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1089/2171] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:38:53 2022] PUT /identity/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:53.475885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.476176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.476457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.506191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] MFA Rules not processed for user `aa0432cd2b5b4d8f8cd2b1ee44e27d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:53.507426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:53.577288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1083/2172] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:38:53.588852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:53.590798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:53.648803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabd86-2a85-4d3f-a0e1-5b2de0245a63 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:53.652491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:53.653083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:53.653362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:53.653635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:53.710601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:53.712073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16e73-d11c-4709-958a-890205048999 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:53.721614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1090/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:58.650709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:58.652247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:58.659124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:58.726485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee44db1-4ea3-4a3b-a550-95f57e4462a7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:58.731447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:58.732085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:58.732507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:58.732816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] PATH_INFO: `/v3/users/7d08548f6ecf4a389ecc23f5a1868e16` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:58.734041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d08548f6ecf4a389ecc23f5a1868e16)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:58.736503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:58.786042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d08548f6ecf4a389ecc23f5a1868e16'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:58.786722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d08548f6ecf4a389ecc23f5a1868e16'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:58.787379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d08548f6ecf4a389ecc23f5a1868e16'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:58.795491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d08548f6ecf4a389ecc23f5a1868e16'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:58.816282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d08548f6ecf4a389ecc23f5a1868e16'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:58.833441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1084/2174] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:38:58 2022] DELETE /identity/v3/users/7d08548f6ecf4a389ecc23f5a1868e16 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:38:58.838710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:58.839207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:58.839341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:58.867280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:38:58.868707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:58.959878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1091/2175] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:38:58.971847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:58.973376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:59.035829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed074022-618e-4999-8ebe-4140a0466ded admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:59.039505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:59.040120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:59.040404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:59.040738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:59.050328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:59.051773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:59.061604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1085/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:38:59.339299 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:38:59.340871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:38:59.357350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aff33-23ab-4eeb-937b-73341fd09449 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:38:59.362859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:38:59.363646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:38:59.364024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:38:59.364444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] PATH_INFO: `/v3/projects/656e9c75b8cf41d5a4cbcb7390eda927` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:38:59.365697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=656e9c75b8cf41d5a4cbcb7390eda927)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:38:59.368345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:38:59.426208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656e9c75b8cf41d5a4cbcb7390eda927', 'request_id': 'req-2362a19c-aa1c-4cea-8b82-636165d5cf69'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:59.428323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:38:59.437812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656e9c75b8cf41d5a4cbcb7390eda927', 'request_id': 'req-2362a19c-aa1c-4cea-8b82-636165d5cf69'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:59.448116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656e9c75b8cf41d5a4cbcb7390eda927', 'request_id': 'req-2362a19c-aa1c-4cea-8b82-636165d5cf69'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:59.460366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] The token cache is being invalidate because project 656e9c75b8cf41d5a4cbcb7390eda927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:38:59.460662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:59.461364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:38:59.463182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1092/2177] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:38:59 2022] DELETE /identity/v3/projects/656e9c75b8cf41d5a4cbcb7390eda927 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:00.120577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:00.122044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:00.152895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:00.180290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677dddd-4719-4a7c-b821-a206d21b6618 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:00.228978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:00.229559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:00.229872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:00.230146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:00.231880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:00.296338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:00.297888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849f8c2-0631-4c6e-8d9c-75ecaf877a3f service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:00.307623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1086/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:07.329983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:07.331568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:07.422901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a19c-aa1c-4cea-8b82-636165d5cf69 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:07.429147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:07.429865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:07.430300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:07.430583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] PATH_INFO: `/v3/users/2de4dd7826144afdb2d7346e416e804f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:07.432042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de4dd7826144afdb2d7346e416e804f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:07.434075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:07.502009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de4dd7826144afdb2d7346e416e804f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:07.502970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de4dd7826144afdb2d7346e416e804f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:07.504176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de4dd7826144afdb2d7346e416e804f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:07.513679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de4dd7826144afdb2d7346e416e804f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:07.528007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de4dd7826144afdb2d7346e416e804f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:07.544882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1093/2179] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:07 2022] DELETE /identity/v3/users/2de4dd7826144afdb2d7346e416e804f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:07.549987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:07.550465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:07.550557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:07.570424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:07.572187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:07.635194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1087/2180] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:07.645129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:07.646621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:07.706187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b0e-a7c6-42f7-b79c-9be29836368e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:07.709822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:07.710430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:07.710714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:07.710991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:07.719847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:07.721281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:07.726176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:07.731184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1094/2181] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:08.003103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.004694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:08.026056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605ed46-6dbf-4b79-95bf-3b0d10eeb151 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.031090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:08.031727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.032015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.032293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] PATH_INFO: `/v3/projects/bec0247c2ba948d8aaf054451db1d978` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.033989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec0247c2ba948d8aaf054451db1d978)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:08.035943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:08.072724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:08.092805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec0247c2ba948d8aaf054451db1d978', 'request_id': 'req-3a0ce157-e329-4ff3-8968-dec368f209c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.099817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec0247c2ba948d8aaf054451db1d978', 'request_id': 'req-3a0ce157-e329-4ff3-8968-dec368f209c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.105672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec0247c2ba948d8aaf054451db1d978', 'request_id': 'req-3a0ce157-e329-4ff3-8968-dec368f209c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.121847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] The token cache is being invalidate because project bec0247c2ba948d8aaf054451db1d978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:08.122146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.122806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.124311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1088/2182] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:08 2022] DELETE /identity/v3/projects/bec0247c2ba948d8aaf054451db1d978 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:08.128383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.138092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8beda2-dbbb-48df-af5d-e2d15837d408 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.198597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:08.199217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.199495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.199763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] PATH_INFO: `/v3/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.200995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0432cd2b5b4d8f8cd2b1ee44e27d7b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:08.203333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:08.248488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0432cd2b5b4d8f8cd2b1ee44e27d7b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.249067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0432cd2b5b4d8f8cd2b1ee44e27d7b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.249667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0432cd2b5b4d8f8cd2b1ee44e27d7b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.258734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0432cd2b5b4d8f8cd2b1ee44e27d7b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.270624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-084ce0b4-e826-4fb6-a574-2567ed7059ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0432cd2b5b4d8f8cd2b1ee44e27d7b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.284017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1095/2183] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:08 2022] DELETE /identity/v3/users/aa0432cd2b5b4d8f8cd2b1ee44e27d7b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:08.570171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.571914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:08.584939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce157-e329-4ff3-8968-dec368f209c4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.641682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:08.642362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.642646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.642918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] PATH_INFO: `/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.644022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e0cd62d91f4863a8e06ba12b6d7d7a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:08.645962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:08.693325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e0cd62d91f4863a8e06ba12b6d7d7a', 'request_id': 'req-ad966008-ee41-4e19-bf89-3b8f42b595c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.700812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e0cd62d91f4863a8e06ba12b6d7d7a', 'request_id': 'req-ad966008-ee41-4e19-bf89-3b8f42b595c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.706649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e0cd62d91f4863a8e06ba12b6d7d7a', 'request_id': 'req-ad966008-ee41-4e19-bf89-3b8f42b595c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.720061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] The token cache is being invalidate because project 54e0cd62d91f4863a8e06ba12b6d7d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:08.720883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.721521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ad966008-ee41-4e19-bf89-3b8f42b595c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:08.723062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1089/2184] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:08 2022] DELETE /identity/v3/projects/54e0cd62d91f4863a8e06ba12b6d7d7a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:08.729555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.730571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.730933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.757093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:08.758404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:08.823925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1096/2185] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:08.833631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e39d3-ff66-410f-bd28-215544597151 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.833971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e39d3-ff66-410f-bd28-215544597151 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.834253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e39d3-ff66-410f-bd28-215544597151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.858782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-911e39d3-ff66-410f-bd28-215544597151 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:08.859992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-911e39d3-ff66-410f-bd28-215544597151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:08.868666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1090/2186] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:08.873396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.883118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dcd69-8bd4-495d-a77a-794a8d73fae6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.887715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:08.888409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.888787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.889115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.890199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:39:08.890517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:08.892123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:08.903939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1097/2187] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:39:08.909001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e39d3-ff66-410f-bd28-215544597151 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.918225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e39d3-ff66-410f-bd28-215544597151 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.923180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:08.924318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:08.924596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:08.924866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:08.925497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:08.927514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:08.962643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1091/2188] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:08 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:08.963769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:08.975049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ccae-f98d-4e48-abac-907c298b6ac2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:08.978762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:09.040792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.041327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.041654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.041961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.042622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.044537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.070664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1098/2189] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:08 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:09.072118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.083708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710ab24-9a84-4f0e-bbe0-abe6ac95029d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.088901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.089666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.089984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.090341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.091028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.092639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.104757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1092/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:09.108142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.118921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f123fc-a311-4d47-bf19-c07d26389bd3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.123584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.124168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.124449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.124723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.125354 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.127660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f00a-cd90-4669-ac20-f149012bbb95 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.138022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1099/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:09.141875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.152872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d70d8-f681-4a7d-86be-019c29705ae4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.157665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.158395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.158683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.158965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] PATH_INFO: `/v3/projects/566512b97339428ab87fce9809119d33/users/062a1f61cf7749bd85cb3309d9b2ac2c/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.161272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:09.182362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] RBAC: Authorizing `identity:create_grant(project_id=566512b97339428ab87fce9809119d33, user_id=062a1f61cf7749bd85cb3309d9b2ac2c, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.187397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.205994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1093/2192] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:09 2022] PUT /identity/v3/projects/566512b97339428ab87fce9809119d33/users/062a1f61cf7749bd85cb3309d9b2ac2c/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:09.216740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.217134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.217509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.246333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] MFA Rules not processed for user `062a1f61cf7749bd85cb3309d9b2ac2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:09.247577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:09.312931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.315653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:09.327424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1100/2193] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:09.341361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.343332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:09.396210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ff59e-99c9-40e2-b2af-fbc32541317b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.402231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.402878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.403166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.403445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] PATH_INFO: `/v3/users/2fab656311164029ac728745e02d80af` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.404610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fab656311164029ac728745e02d80af)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.407333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.460252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a397-c9d5-427f-8d63-a044e8cc7aea None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.464429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.465129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.465719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.466511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.475661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.476624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fab656311164029ac728745e02d80af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:09.477535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b8b4f-416d-4d44-b7dc-f43cfae35fd4 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.478407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fab656311164029ac728745e02d80af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:09.479172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fab656311164029ac728745e02d80af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:09.482312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:09.490714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fab656311164029ac728745e02d80af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:09.510377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1101/2194] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:39:09.532578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fab656311164029ac728745e02d80af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:09.556195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1094/2195] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:09 2022] DELETE /identity/v3/users/2fab656311164029ac728745e02d80af => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:39:09.560699 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.561063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.561458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.586303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:09.587789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:09.662655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1102/2196] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:39:09.672304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:09.674618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:09.739594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd22d-e934-4043-b1c2-61ee968353c7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:09.744811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:09.745566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:09.745900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:09.746168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:09.756408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:09.758230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d11d7-d697-4a2a-8f04-21a056dd2c14 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:09.768248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1095/2197] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:10.108553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.110362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:10.125198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afeafb-69ce-47e3-83f6-c79607acc2b6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.129961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.130600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.131372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.131372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] PATH_INFO: `/v3/projects/5b0a4831f37e4e199d943c3f2474581d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.132382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0a4831f37e4e199d943c3f2474581d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.134422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.187564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0a4831f37e4e199d943c3f2474581d', 'request_id': 'req-4f978661-67d4-4070-ac79-f7430eba73a0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:10.196741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0a4831f37e4e199d943c3f2474581d', 'request_id': 'req-4f978661-67d4-4070-ac79-f7430eba73a0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:10.203170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0a4831f37e4e199d943c3f2474581d', 'request_id': 'req-4f978661-67d4-4070-ac79-f7430eba73a0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:10.214260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] The token cache is being invalidate because project 5b0a4831f37e4e199d943c3f2474581d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:10.214552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:10.215213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4f978661-67d4-4070-ac79-f7430eba73a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:10.216728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1103/2198] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:10 2022] DELETE /identity/v3/projects/5b0a4831f37e4e199d943c3f2474581d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:10.221686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.222000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.222291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.234065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:10.251132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:10.253135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:10.315175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1096/2199] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:10.324439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.324728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.325012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.345330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:10.346690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:10.354836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1104/2200] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:39:10.359087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.372404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d6652-054e-477a-b73a-bb971ded4366 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.378215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.378855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.379143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.379478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.380145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:39:10.380455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.382158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.395564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1097/2201] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:39:10.400165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.409960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57f02-3c3a-4ca3-9aca-0206466274eb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.414625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.415250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.415530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.415802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.416470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.418427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.446711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1105/2202] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:10 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:10.450992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.461128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3389d-fe4f-4f2d-be88-f7abd93f96e3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.518953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.519576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.519866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.520144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.520794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.522808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.550570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1098/2203] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:10 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:39:10.554966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.568515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56613b99-b563-42ab-b282-a2e98d224cd5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.578227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.578834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.579120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.579396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.580037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.581743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.592502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1106/2204] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:39:10.596198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.606509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccb22-9af1-416b-9111-2ab9609bc1e9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.612218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.612455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.612743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.613019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.613697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.615462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68296391-8932-4eb6-ad1b-611a0aecaf55 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.635240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1099/2205] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:10.639659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.651475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd212-585a-4761-a0ea-b80c8101e5df admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.662233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.662890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.663175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.663452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] PATH_INFO: `/v3/projects/5fe7febcf66545ed971a82ec6df500b1/users/6e55335d4c8f4a8abf67752192602a31/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.678525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe7febcf66545ed971a82ec6df500b1, user_id=6e55335d4c8f4a8abf67752192602a31, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.681210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.694261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1107/2206] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:10 2022] PUT /identity/v3/projects/5fe7febcf66545ed971a82ec6df500b1/users/6e55335d4c8f4a8abf67752192602a31/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:10.705681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7214-5865-4e4b-8d8f-14e16281b5bb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.706332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7214-5865-4e4b-8d8f-14e16281b5bb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.706572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7214-5865-4e4b-8d8f-14e16281b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.744309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-af7e7214-5865-4e4b-8d8f-14e16281b5bb None None] MFA Rules not processed for user `6e55335d4c8f4a8abf67752192602a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:10.745522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-af7e7214-5865-4e4b-8d8f-14e16281b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:10.804788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:10.809215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1100/2207] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:10.820562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:10.822488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:10.846393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:10.887281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dac38-b0a0-4aa9-8928-f27b2f2fe4a9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:10.891052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:10.891657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:10.891944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:10.892248 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:10.900324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:10.901772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:10.910847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1108/2208] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:20.906463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=18 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:25.313025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7f0f-e5cf-472e-b6c6-5d36f8a0e3ac None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:25.313579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7f0f-e5cf-472e-b6c6-5d36f8a0e3ac None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:25.313693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb7f0f-e5cf-472e-b6c6-5d36f8a0e3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:25.333422 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-88eb7f0f-e5cf-472e-b6c6-5d36f8a0e3ac None None] MFA Rules not processed for user `d718e35636464834bc20a6eda0599cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:25.334673 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-88eb7f0f-e5cf-472e-b6c6-5d36f8a0e3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:25.394200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1101/2209] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:35.387113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=25 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:39.243567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:39.245249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:39.311736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91fcec-7e9c-4bff-89b5-021b19d256ec service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:39.316235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:39.316855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:39.317137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:39.317504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] PATH_INFO: `/v3/users/062a1f61cf7749bd85cb3309d9b2ac2c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:39.319326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=062a1f61cf7749bd85cb3309d9b2ac2c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:39.322057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:39.377864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062a1f61cf7749bd85cb3309d9b2ac2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:39.378538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062a1f61cf7749bd85cb3309d9b2ac2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:39.379575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062a1f61cf7749bd85cb3309d9b2ac2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:39.387030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062a1f61cf7749bd85cb3309d9b2ac2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:39.402428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062a1f61cf7749bd85cb3309d9b2ac2c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:39.419434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1109/2210] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:39 2022] DELETE /identity/v3/users/062a1f61cf7749bd85cb3309d9b2ac2c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:39.424537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:39.425068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:39.425301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:39.447346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:39.448575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:39.516532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1102/2211] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:39.526944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:39.528376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:39.593188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c40c-9832-4c65-9680-a6797a14b38f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:39.597234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:39.597907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:39.598580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:39.598865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:39.609188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:39.610870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5874376-5848-483c-a132-852dc74ff1b8 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:39.620899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1110/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:39.965970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:39.967436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:39.981439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf116d-cb91-48d1-b807-1eca5ca889a2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:39.993900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:39.994737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:39.995237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:39.995482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] PATH_INFO: `/v3/projects/566512b97339428ab87fce9809119d33` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:39.996937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566512b97339428ab87fce9809119d33)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:39.999350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.063615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566512b97339428ab87fce9809119d33', 'request_id': 'req-bd795870-ad5e-401f-b374-9fec2c49ae56'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:40.072105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566512b97339428ab87fce9809119d33', 'request_id': 'req-bd795870-ad5e-401f-b374-9fec2c49ae56'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:40.080601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566512b97339428ab87fce9809119d33', 'request_id': 'req-bd795870-ad5e-401f-b374-9fec2c49ae56'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:40.104024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] The token cache is being invalidate because project 566512b97339428ab87fce9809119d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:40.104024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:40.104816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-bd795870-ad5e-401f-b374-9fec2c49ae56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:40.112193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1103/2213] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:39 2022] DELETE /identity/v3/projects/566512b97339428ab87fce9809119d33 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:40.113237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.113866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.114051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.150241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:40.151514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:40.180710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:40.252296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1111/2214] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:39:40.261669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.262143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.262425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.283503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:40.284734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:40.293255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1104/2215] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:40.298095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.309395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bf493-8f1e-4c97-80cc-b319a40d239b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.314149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.314760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.315047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.315325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.315986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:39:40.316324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.318033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.335847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1112/2216] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 15:39:40.340238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.358401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb616b98-a6f3-4593-a4b8-9168cbbba335 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.374529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.375166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.375535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.375809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.376523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.378521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.408552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1105/2217] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:40.412913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.423599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe30cf3-d67c-41a8-9e39-84a8ac1cfa05 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.495042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.495673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.495963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.496246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.496917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.498935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.533038 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1113/2218] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:40.537070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.548121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cee75f-b66f-48e4-97f0-eaf1ac08d043 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.554307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.554974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.555262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.555536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.556213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.558193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.570855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1106/2219] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:40.575125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.585580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7ffd-23af-42ab-9497-64cfbf5e8030 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.590591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.591289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.591682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.592048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.592759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.594497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db959d6-d0ed-4c7e-9495-aace7ddefa3f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.608695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1114/2220] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:40.612991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.624479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d69b7-de10-4f81-9c40-5fff619f29a6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.629372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.630089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.630379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.630698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] PATH_INFO: `/v3/projects/85b7e2fd0c204e5fb97ea497b81616fc/users/55dc070358da4a82a64932407ad16f58/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.655387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b7e2fd0c204e5fb97ea497b81616fc, user_id=55dc070358da4a82a64932407ad16f58, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.658216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.676409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1107/2221] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:40 2022] PUT /identity/v3/projects/85b7e2fd0c204e5fb97ea497b81616fc/users/55dc070358da4a82a64932407ad16f58/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:40.688046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.688342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.688845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.725432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] MFA Rules not processed for user `55dc070358da4a82a64932407ad16f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:40.726629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:40.810538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1115/2222] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:40.815154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.846283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:40.901639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb56c3-6590-4596-ab86-e0155acc95bc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.906572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:40.907383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:40.907688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:40.907966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:40.909939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:40.912601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:40.953082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1108/2223] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:40.957269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:40.970286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933353d8-ec96-4db6-8af4-29438ca16bdd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:40.989955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:41.036728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.037916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.038198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.038464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.039095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.041020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.065637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1116/2224] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:40 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:41.066914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.078405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d08489-a4fd-4797-ae8a-4eba27855c7b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.085373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.086028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.086315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.086608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.087260 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.088960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.105866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1109/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:41.107006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.122522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3de45-8ad8-4053-9c6e-758ba05fbb77 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.134192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.135023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.135294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.135575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.136720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.138795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3706991-4df7-4fe1-bae3-4b2bdadc098e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.154075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1117/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:41.156104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.171576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91c760-dc67-4035-bcd2-378b9b8057fd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.176815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.177741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.177858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.178235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] PATH_INFO: `/v3/projects/08f83604193b4242b3d55ff53eb62141/users/db45b26f717449ac903a90c1b42d3bd4/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.199184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f83604193b4242b3d55ff53eb62141, user_id=db45b26f717449ac903a90c1b42d3bd4, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.202420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.225730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1110/2227] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:41 2022] PUT /identity/v3/projects/08f83604193b4242b3d55ff53eb62141/users/db45b26f717449ac903a90c1b42d3bd4/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:41.234702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.234915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.240546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.270615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] MFA Rules not processed for user `db45b26f717449ac903a90c1b42d3bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:41.272635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:41.352501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1118/2228] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:41.356736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.438378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5af6b-789a-4388-a22f-267e2d7912ea admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.445801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.445801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.446199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.446199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.447200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.451860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.502186 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1111/2229] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:41 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:39:41.506787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.518294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4a955-c548-4b87-ab93-f23729f45272 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.665200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.665872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.666224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.666569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.667276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.669250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635a678-94ce-40b0-b64a-7afeb945ccce admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.691571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1119/2230] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:41 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:41.696182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.707421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72bae7-de44-4c81-89c1-6dcf46033fd5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.721154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.721385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.721503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.721720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.721820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.722102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.722398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.723078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.724742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.737644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1112/2231] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:41.741647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.746609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] MFA Rules not processed for user `1ca7bef620284345a977088f9a0d2a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:41.747832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:41.751892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbd200-9793-40d5-8b3d-dcb3d497d50a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.760360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.760582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:41.761038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.761592 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.761967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] PATH_INFO: `/v3/projects/f3392969adf74d52ab923dd8bde334c3/users/8f986af66f64438b921e34051b8cd15f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.778103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3392969adf74d52ab923dd8bde334c3, user_id=8f986af66f64438b921e34051b8cd15f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.780773 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.793425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1113/2232] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:41 2022] PUT /identity/v3/projects/f3392969adf74d52ab923dd8bde334c3/users/8f986af66f64438b921e34051b8cd15f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:41.797488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.814189 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1120/2233] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:39:41.820155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:41.866205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd301d9-25cf-4b13-83c9-9720d51e6c06 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.872137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.872801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.873089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.873371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.874069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.875734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0995-12e9-4a7d-b03b-2510337ca286 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.893317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1114/2234] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:39:41.894540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:41.906343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e461a5-620f-4287-9fde-7f5ff9ca7cfb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:41.911280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:41.911949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.912256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.912705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] PATH_INFO: `/v3/projects/f3392969adf74d52ab923dd8bde334c3/users/8f986af66f64438b921e34051b8cd15f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.917379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3392969adf74d52ab923dd8bde334c3, user_id=8f986af66f64438b921e34051b8cd15f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:41.920135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:41.941753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1121/2235] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:41 2022] PUT /identity/v3/projects/f3392969adf74d52ab923dd8bde334c3/users/8f986af66f64438b921e34051b8cd15f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:39:41.947587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:41.947880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:41.948591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:41.983199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] MFA Rules not processed for user `8f986af66f64438b921e34051b8cd15f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:41.984807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:42.098879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1115/2236] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:39:42.109012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:42.110842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:42.214169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c76eb-a789-43e8-a4e8-cb8753fcbd90 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:42.219866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:42.220694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:42.221052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:42.222147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:42.282836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:42.284196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:42.297855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1122/2237] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:43.934895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:43.936339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:43.950122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dae2c-d88f-4ac6-8d84-c4eb73fb9e51 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:43.953883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:43.954503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:43.954775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:43.955052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:44.016378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:44.018015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:44.030812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1116/2238] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:44.651767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:44.660507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d74c02-e471-4ee8-b3b9-12cdba47a24f service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:44.664737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:44.665364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:44.665655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:44.665968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:44.675606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:44.677052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049dbc0-fe15-470c-80f8-ff804866c9c3 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:44.687123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1123/2239] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:48.702882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:48.799807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:48.846022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda73321-7934-4fc0-a564-958a5462f548 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:48.854293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:48.855054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:48.855443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:48.855977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] PATH_INFO: `/v3/users/55dc070358da4a82a64932407ad16f58` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:48.857272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dc070358da4a82a64932407ad16f58)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:48.860065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:48.944492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dc070358da4a82a64932407ad16f58'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:48.945231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dc070358da4a82a64932407ad16f58'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:48.945904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dc070358da4a82a64932407ad16f58'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:48.956215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dc070358da4a82a64932407ad16f58'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:48.975671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dc070358da4a82a64932407ad16f58'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:49.004667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1117/2240] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:48 2022] DELETE /identity/v3/users/55dc070358da4a82a64932407ad16f58 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:49.009101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:49.009649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:49.009649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:49.060202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:49.062207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:49.207452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:49.234166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1124/2241] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:49.249741 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:49.251230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:49.339449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4af8b-1731-4b3d-8e38-6048ee767701 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:49.344183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:49.344810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:49.345095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:49.345371 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:49.356122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:49.357914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:49.376542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1118/2242] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:50.033988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:50.038401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:50.055991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3673c-3bbc-4814-b403-da089ac957da None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:50.062228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:50.063428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:50.063780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:50.064253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] PATH_INFO: `/v3/projects/85b7e2fd0c204e5fb97ea497b81616fc` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:50.066938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b7e2fd0c204e5fb97ea497b81616fc)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:50.070204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:50.169015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b7e2fd0c204e5fb97ea497b81616fc', 'request_id': 'req-1998fdc8-8624-4be3-b510-e9495f49d539'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.189978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b7e2fd0c204e5fb97ea497b81616fc', 'request_id': 'req-1998fdc8-8624-4be3-b510-e9495f49d539'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.198176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b7e2fd0c204e5fb97ea497b81616fc', 'request_id': 'req-1998fdc8-8624-4be3-b510-e9495f49d539'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.235948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] The token cache is being invalidate because project 85b7e2fd0c204e5fb97ea497b81616fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:50.236678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.237417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.239133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1125/2243] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:50 2022] DELETE /identity/v3/projects/85b7e2fd0c204e5fb97ea497b81616fc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:50.243215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:50.261223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4acf-7105-4d0f-96f2-1b90210a25f9 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:50.338193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:50.339338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:50.339338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:50.339523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] PATH_INFO: `/v3/users/db45b26f717449ac903a90c1b42d3bd4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:50.341163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=db45b26f717449ac903a90c1b42d3bd4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:50.342676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:50.345299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:50.398496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db45b26f717449ac903a90c1b42d3bd4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.399159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db45b26f717449ac903a90c1b42d3bd4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.399774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db45b26f717449ac903a90c1b42d3bd4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.408811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db45b26f717449ac903a90c1b42d3bd4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.422936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db45b26f717449ac903a90c1b42d3bd4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:50.437352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1119/2244] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:50 2022] DELETE /identity/v3/users/db45b26f717449ac903a90c1b42d3bd4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:50.865314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:50.867375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:50.881095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998fdc8-8624-4be3-b510-e9495f49d539 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:50.964207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:50.964862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:50.965126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:50.965405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] PATH_INFO: `/v3/projects/08f83604193b4242b3d55ff53eb62141` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:50.966554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f83604193b4242b3d55ff53eb62141)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:50.968469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.016871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f83604193b4242b3d55ff53eb62141', 'request_id': 'req-a8a0e78c-f02b-465f-a789-bef25e95ff82'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.024183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f83604193b4242b3d55ff53eb62141', 'request_id': 'req-a8a0e78c-f02b-465f-a789-bef25e95ff82'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.030321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f83604193b4242b3d55ff53eb62141', 'request_id': 'req-a8a0e78c-f02b-465f-a789-bef25e95ff82'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.040581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] The token cache is being invalidate because project 08f83604193b4242b3d55ff53eb62141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:51.041165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.041466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.042959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1126/2245] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:50 2022] DELETE /identity/v3/projects/08f83604193b4242b3d55ff53eb62141 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:51.047261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:51.048377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:51.061583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35fd8-2527-4de0-9471-3a1d54bf3ddf admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:51.119759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:51.120315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.120599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.120876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] PATH_INFO: `/v3/users/8f986af66f64438b921e34051b8cd15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.122027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f986af66f64438b921e34051b8cd15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:51.123961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.170447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f986af66f64438b921e34051b8cd15f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.171103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f986af66f64438b921e34051b8cd15f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.171738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f986af66f64438b921e34051b8cd15f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.179483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f986af66f64438b921e34051b8cd15f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.191955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b48c694-3415-42c3-b930-5806a7a98a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f986af66f64438b921e34051b8cd15f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.201809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:51.207124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1120/2246] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:39:51 2022] DELETE /identity/v3/users/8f986af66f64438b921e34051b8cd15f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:51.483863 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:51.485289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:51.493990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:51.501229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0e78c-f02b-465f-a789-bef25e95ff82 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:51.569494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:51.570199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.570510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.570783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] PATH_INFO: `/v3/projects/f3392969adf74d52ab923dd8bde334c3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.572188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3392969adf74d52ab923dd8bde334c3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:51.574152 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.622079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3392969adf74d52ab923dd8bde334c3', 'request_id': 'req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.629470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3392969adf74d52ab923dd8bde334c3', 'request_id': 'req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.635955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3392969adf74d52ab923dd8bde334c3', 'request_id': 'req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.646682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] The token cache is being invalidate because project f3392969adf74d52ab923dd8bde334c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:39:51.647518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.648194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5f21bb10-de63-40c0-bb6e-df6b13fb4e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:39:51.651123 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1127/2247] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:39:51 2022] DELETE /identity/v3/projects/f3392969adf74d52ab923dd8bde334c3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:39:51.656559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.657065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.657171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.679943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:51.681178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:51.749716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1121/2248] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:51.758986 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.759271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.759547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.779810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:51.781017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:51.789436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1128/2249] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:51.793729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:51.803757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c180222-3602-4cd0-8e57-5e431f5b3815 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:51.808378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:51.808982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.809264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.809539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.810220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 15:39:51.810529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:51.812247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.824029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1122/2250] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 15:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 15:39:51.828366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:51.838110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0bdc6-1bd0-4959-a1eb-d66364802e6a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:51.842750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:51.843339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.843620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.843890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.844529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:51.846450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.882372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1129/2251] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 15:39:51 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 15:39:51.886450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:51.896273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265c503-52e2-4a1a-bbfc-98b26fb64690 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:51.959013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:51.961679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:51.962003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:51.962284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:51.962945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:51.964890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:51.998870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1123/2252] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 15:39:51 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 15:39:52.001918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:52.013026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f77fb-6791-4ef1-ab38-c33c390ac12e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:52.018667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:52.019365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:52.019780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:52.020051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:52.020761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:52.022520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:52.036681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1130/2253] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 15:39:52.038483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:52.048783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da6137-8e59-42a3-8f4b-6cd396876210 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:52.054695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:52.055328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:52.055609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:52.055905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:52.056554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:52.058199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c33dc-d861-44a5-bc21-ce2a471f1655 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:52.073797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1124/2254] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 15:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 15:39:52.075357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:52.086403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e06a8-8dab-43c4-b450-b574d35f3bb3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:52.098184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:52.100928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:52.101512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:52.102129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] PATH_INFO: `/v3/projects/c8dd61b0d73641ee8df5d5fddc0b335a/users/af2aea36cdb84e2ca75783b7095e0752/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:52.124181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dd61b0d73641ee8df5d5fddc0b335a, user_id=af2aea36cdb84e2ca75783b7095e0752, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:52.127083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:52.142210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1131/2255] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 15:39:52 2022] PUT /identity/v3/projects/c8dd61b0d73641ee8df5d5fddc0b335a/users/af2aea36cdb84e2ca75783b7095e0752/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 15:39:52.155629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1627-fa5c-468c-a004-0eba73b0bace None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:52.156078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1627-fa5c-468c-a004-0eba73b0bace None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:52.156544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed1627-fa5c-468c-a004-0eba73b0bace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:52.189797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-31ed1627-fa5c-468c-a004-0eba73b0bace None None] MFA Rules not processed for user `af2aea36cdb84e2ca75783b7095e0752`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:52.191101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-31ed1627-fa5c-468c-a004-0eba73b0bace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:52.249006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:52.261943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1125/2256] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:52.269438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:52.271201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:52.304074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:39:52.332568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f924-94d2-457c-9141-d757ccf57bdb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:52.336849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:52.337465 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:52.337781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:52.338066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:52.346349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:52.348060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:52.357595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1132/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:39:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:39:59.296065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5869-cb0c-416c-aaa5-cc522032bfcf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:59.296585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5869-cb0c-416c-aaa5-cc522032bfcf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:59.296690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe5869-cb0c-416c-aaa5-cc522032bfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:59.320103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a9fe5869-cb0c-416c-aaa5-cc522032bfcf None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:39:59.321343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a9fe5869-cb0c-416c-aaa5-cc522032bfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:59.399150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1126/2258] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:39:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:39:59.461244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:39:59.467378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:39:59.560813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fcd-f7c8-40d1-9019-4d31d7383313 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:39:59.567169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:39:59.567864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:39:59.568243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:39:59.568599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:39:59.578706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:39:59.580617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:39:59.590237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1133/2259] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 15:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 15:40:06.910997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:06.911680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:06.911896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:06.937265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] MFA Rules not processed for user `10a5c8ccdc2b49d68a6c088207d593d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:06.940185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:07.008616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1127/2260] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:40:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:40:09.587043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=45 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:10.098154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:10.099780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:10.183419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430806ba-4dca-430f-b17d-033b1e6a8b30 service swift] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:10.188968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:10.189964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:10.190311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:10.190594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:10.192450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:10.205612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:10.207666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715689e-564b-4abe-9350-71ef813afbc1 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:10.218552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1134/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:40:14.870665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:14.872787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:14.971402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9751e-310d-4190-adaf-ef69b58c4553 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:14.976890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:14.977677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:14.978744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:14.979183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] PATH_INFO: `/v3/users/6e55335d4c8f4a8abf67752192602a31` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:14.981106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e55335d4c8f4a8abf67752192602a31)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:14.984133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:15.081971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e55335d4c8f4a8abf67752192602a31'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:15.083129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e55335d4c8f4a8abf67752192602a31'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:15.084070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e55335d4c8f4a8abf67752192602a31'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:15.086516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:15.104611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e55335d4c8f4a8abf67752192602a31'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:15.154936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e55335d4c8f4a8abf67752192602a31'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:15.197241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1128/2262] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:40:14 2022] DELETE /identity/v3/users/6e55335d4c8f4a8abf67752192602a31 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:40:15.202117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:15.202608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:15.202799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:15.234335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:15.236542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:15.338015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1135/2263] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:40:15.348640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:15.350829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:15.440432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf345-f9dd-4dd9-815a-6548b2fb2b82 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:15.446657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:15.447589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:15.447876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:15.448213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:15.461336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:15.462883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:15.473744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1129/2264] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:40:16.066856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:16.069274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:16.097731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991813-abbd-41b4-a2b2-d5a35985c0ca None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:16.113720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:16.116380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.117085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.117734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] PATH_INFO: `/v3/projects/5fe7febcf66545ed971a82ec6df500b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.119707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe7febcf66545ed971a82ec6df500b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:16.122683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:16.199504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe7febcf66545ed971a82ec6df500b1', 'request_id': 'req-b75fe74c-b001-44ad-b848-6fda4162cef6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.228964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe7febcf66545ed971a82ec6df500b1', 'request_id': 'req-b75fe74c-b001-44ad-b848-6fda4162cef6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.248046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe7febcf66545ed971a82ec6df500b1', 'request_id': 'req-b75fe74c-b001-44ad-b848-6fda4162cef6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.265642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] The token cache is being invalidate because project 5fe7febcf66545ed971a82ec6df500b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:40:16.265642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.266215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b75fe74c-b001-44ad-b848-6fda4162cef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.267990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1136/2265] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:40:16 2022] DELETE /identity/v3/projects/5fe7febcf66545ed971a82ec6df500b1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:40:16.335920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:16.337563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:16.359701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851142a-edab-419a-ad44-6a178059ddb2 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:16.463920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:16.464870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.465434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.466092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] PATH_INFO: `/v3/users/af2aea36cdb84e2ca75783b7095e0752` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.467917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2aea36cdb84e2ca75783b7095e0752)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:16.469938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:16.528617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2aea36cdb84e2ca75783b7095e0752'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.529511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2aea36cdb84e2ca75783b7095e0752'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.530694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2aea36cdb84e2ca75783b7095e0752'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.540302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2aea36cdb84e2ca75783b7095e0752'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.553648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:16.556481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2aea36cdb84e2ca75783b7095e0752'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:16.588647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1130/2266] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 15:40:16 2022] DELETE /identity/v3/users/af2aea36cdb84e2ca75783b7095e0752 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:40:16.592969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2dea1-cb72-4f6f-a580-7e297e15c542 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.593260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2dea1-cb72-4f6f-a580-7e297e15c542 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.593542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2dea1-cb72-4f6f-a580-7e297e15c542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.615965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-21a2dea1-cb72-4f6f-a580-7e297e15c542 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:16.617273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-21a2dea1-cb72-4f6f-a580-7e297e15c542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:16.627420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:16.689842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1137/2267] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:40:16.699782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:16.701925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:16.777588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253068f1-3172-4c72-baf3-f3c3353595e8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:16.781845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:16.782522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.782809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.783104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.797553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:16.799228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39011-bd46-4932-b23c-979f1ca437f2 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:16.810132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1131/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:40:16.897215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.897508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.897841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.899225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1138/2269] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:40:16.907732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:16.908009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:16.908264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:16.931637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:16.932944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.004436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1132/2270] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:40:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:40:17.018539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:17.020533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.089659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664b22-8b00-452a-80c8-f07fbea7cb7b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:17.093969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:17.094584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.094864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.095166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.106451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:17.107879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98c280-41b2-4e5d-b5b2-a53a88fe8227 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:17.119281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1139/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:40:17.179111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:17.181641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.191989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b98b1-2e99-4602-b179-9af9bffa1074 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.192340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b98b1-2e99-4602-b179-9af9bffa1074 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.192624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b98b1-2e99-4602-b179-9af9bffa1074 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.194142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1140/2272] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 15:40:17.199367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.199910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.200245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.200477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8770578-06ed-49e2-b36a-b657bc984f4e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:17.205913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:17.206564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.206956 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.207189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] PATH_INFO: `/v3/projects/c8dd61b0d73641ee8df5d5fddc0b335a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.208295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dd61b0d73641ee8df5d5fddc0b335a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:17.210858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:17.222925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] MFA Rules not processed for user `0621ba7d8a164789958d1cadba19ea6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:17.224207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.231505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1141/2273] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:40:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:40:17.243242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:17.245470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.257559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02c868-b8db-499d-be4f-5ba4930277be None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:17.267604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dd61b0d73641ee8df5d5fddc0b335a', 'request_id': 'req-065a8365-38b7-4ef6-b940-4b928221d1c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:17.274764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dd61b0d73641ee8df5d5fddc0b335a', 'request_id': 'req-065a8365-38b7-4ef6-b940-4b928221d1c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:17.281305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dd61b0d73641ee8df5d5fddc0b335a', 'request_id': 'req-065a8365-38b7-4ef6-b940-4b928221d1c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:17.297889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] The token cache is being invalidate because project c8dd61b0d73641ee8df5d5fddc0b335a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 15:40:17.298328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:17.299178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-065a8365-38b7-4ef6-b940-4b928221d1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 15:40:17.300895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1133/2274] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 15:40:17 2022] DELETE /identity/v3/projects/c8dd61b0d73641ee8df5d5fddc0b335a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 15:40:17.327108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:17.327749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.328026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.328297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.331168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.344264 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:17.345860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e239b-c69e-4ce2-80bf-9526b1a2fd3d service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:17.358039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1142/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:40:17.447024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.447574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.447574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.449082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1134/2276] 10.222.0.20 () {54 vars in 921 bytes} [Mon Sep 12 15:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 15:40:17.454358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a47f9d-b362-4d4c-a975-b2178c03c8fa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.454647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a47f9d-b362-4d4c-a975-b2178c03c8fa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.454925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a47f9d-b362-4d4c-a975-b2178c03c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.476504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a9a47f9d-b362-4d4c-a975-b2178c03c8fa None None] MFA Rules not processed for user `c615724d4e5944169ead59e30e0d04b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:17.477788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a9a47f9d-b362-4d4c-a975-b2178c03c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.527695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:17.547710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1143/2277] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Sep 12 15:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:40:17.558133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:17.559866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:17.569436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:17.574627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5728cd-c09a-4099-b9bb-d90e9f25d8c6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:17.579426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:17.580024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:17.580310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:17.580585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:17.591431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:17.592937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:17.603657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1135/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:40:20.253535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be07dc-2d19-4568-b653-4289761ea969 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:20.254134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be07dc-2d19-4568-b653-4289761ea969 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:20.254134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be07dc-2d19-4568-b653-4289761ea969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:20.274036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-41be07dc-2d19-4568-b653-4289761ea969 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:40:20.275261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-41be07dc-2d19-4568-b653-4289761ea969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:20.329765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1144/2279] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 15:40:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:40:20.341646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:40:20.343058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:40:20.411811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da13d1f-c1e5-4a64-ac43-3f3025dfbd27 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:40:20.417413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:40:20.418053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:40:20.418341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:40:20.418621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:40:20.426696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:40:20.428903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:40:20.436856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1136/2280] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 15:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 15:40:30.323000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:40:30.433665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=36 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:41:35.392991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7b8b-839f-40f3-b72c-f6b1a3ff94e4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:41:35.393498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7b8b-839f-40f3-b72c-f6b1a3ff94e4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:41:35.393610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7b8b-839f-40f3-b72c-f6b1a3ff94e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:41:35.414099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ea4b7b8b-839f-40f3-b72c-f6b1a3ff94e4 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:41:35.415813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea4b7b8b-839f-40f3-b72c-f6b1a3ff94e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:41:35.423111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1145/2281] 10.222.0.20 () {58 vars in 967 bytes} [Mon Sep 12 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:41:35.464997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:41:35.476000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731185-16fa-4d56-be68-c3cd9ae37529 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:41:35.487569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:41:35.488691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:41:35.488986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:41:35.489653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:41:35.501119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:41:35.502869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:41:35.511494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1137/2282] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 15:41:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 15:41:45.407635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:41:45.507975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:41:54.900768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:41:54.901262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:41:54.901262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:41:54.919771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] MFA Rules not processed for user `d718e35636464834bc20a6eda0599cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:41:54.920963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:41:54.974674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1146/2283] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 15:41:58.803605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:41:58.805647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:41:58.817266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bba9-5b84-4d96-9601-a362d15bae77 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:41:58.820852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:41:58.821435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:41:58.821730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:41:58.822024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:41:58.823938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:41:58.836976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:41:58.838438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:41:58.849128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1138/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:04.968036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=25 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:42:08.844304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:42:16.768921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:16.770390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:16.787097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3281a7-1974-4f01-b3a2-5aee4e6befbd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:16.792577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:16.793326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:16.794014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:16.794014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:16.795679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:16.855212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:16.856845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:16.866488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1147/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:18.712835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:18.714313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:18.772325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15e646-2d03-4e67-992e-685e5e68c45b service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:18.777225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:18.777789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:18.778077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:18.778353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:18.780080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:18.838917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:18.840365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:18.849220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1139/2286] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:20.348238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:20.358002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f791f3-36ca-4eed-a070-8f3f6a6616a7 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:20.361572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:20.362206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:20.362494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:20.362773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:20.364543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:20.378717 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:20.380142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae143175-2bd2-4f76-aa50-b114ed6544dc service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:20.391021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1148/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:22.718729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:22.732460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e63ad-87cb-4073-926d-6789e5fff0b0 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:22.739166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:22.740034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:22.740479 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:22.740920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:22.743027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:22.813196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:22.815081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:22.825936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1140/2288] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:27.033199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:27.033757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:27.033757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:27.053409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] MFA Rules not processed for user `5d1ab08e56ca4ad692869347a8cd7e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:42:27.054641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:27.061449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1149/2289] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:42:27.369627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:27.378866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5363d0-b50d-4426-8041-b4a8ef86b2c4 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:27.382389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:27.382991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:27.383275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:27.383550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:27.385758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:27.394929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:27.396339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:27.404812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1141/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:27.722487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:27.724026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:27.794422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8bd7-b408-4f7c-91f3-fa30e6507da6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:27.799436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:27.800043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:27.800328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:27.800606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:27.802365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:27.813985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:27.815580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:27.825938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1150/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:32.421844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:42:32.432140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa161-01f7-4bc4-aab3-a50e24f28591 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:42:32.436502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:42:32.437180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:42:32.437532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:42:32.437921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:42:32.439784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:42:32.451611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:42:32.453073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:42:32.462302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1142/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:42:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:42:37.821048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=69 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:42:42.458008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:43:10.979292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:43:10.989027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc142f6-8ced-415a-bf01-af596ab399f1 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:43:10.992557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:43:10.993143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:43:10.993422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:43:10.993691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:43:10.995451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:43:11.004641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:43:11.006180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:43:11.013833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1151/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:43:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:43:20.673682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:43:20.682511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbba79-36da-47ec-a83a-64d320c1e4d8 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:43:20.686180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:43:20.686777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:43:20.687062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:43:20.687339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:43:20.689044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:43:20.747200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:43:20.748642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:43:20.757492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1143/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:43:21.010399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:43:30.753316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=28 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:45:16.371003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:45:16.433622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cec3-f2b3-430e-9e5a-49bbaed4b130 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:45:16.439364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:45:16.439958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:45:16.440257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:45:16.440508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:45:16.442232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:45:16.474642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:45:16.476204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:45:16.486524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1152/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:45:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:45:26.481790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=36 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:47:16.170173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:47:16.171495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:47:16.248452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df3197-bef0-43ce-9974-324acaf180ac service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:47:16.254073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:47:16.254673 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:47:16.254955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:47:16.255226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:47:16.263269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:47:16.264834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:47:16.294782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1144/2296] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:47:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:47:18.034629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:47:18.053053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bec95-ee55-4d85-9348-b8521fde9eaf service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:47:18.056989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:47:18.057695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:47:18.058103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:47:18.058456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:47:18.067526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:47:18.068950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:47:18.098491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1153/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:47:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:47:22.122455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:47:22.133509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae5fdf-d5fe-4522-bcdc-27b6e00cd08d service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:47:22.139098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:47:22.139691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:47:22.139975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:47:22.140251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:47:22.148020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:47:22.149601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:47:22.177444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1145/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:47:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:47:28.094850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=17 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:47:31.800393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:47:31.812618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c3009-aa7e-4e34-a2ae-e77bfc8ae846 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:47:31.818877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:47:31.819471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:47:31.819755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:47:31.820027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:47:31.828269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:47:31.830594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:47:31.857517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1154/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:47:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:47:32.174781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=53 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:47:41.854829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=13 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:48:16.983113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:48:17.003451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d74686-5fe4-4d25-99e5-dbd4f8a3681f service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:48:17.007071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:48:17.007668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:48:17.007946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:48:17.008216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:48:17.015281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:48:17.016733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:48:17.024534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1146/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:48:21.842671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:48:21.844135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:48:21.869070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ae164-d851-40ff-8fb4-615aaf0a0b80 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:48:21.872766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:48:21.873501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:48:21.873972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:48:21.874251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:48:21.883480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:48:21.885006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:48:21.915763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1155/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:48:27.021045 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=10 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:48:31.912460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=18 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:49:15.734056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:49:15.745974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dd205-53e6-4211-8b98-9c05ea28c6b9 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:49:15.749662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:49:15.750842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:49:15.751157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:49:15.751435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:49:15.761721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:49:15.763150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:49:15.773868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1147/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:49:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:49:16.908553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:49:16.923776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061288be-71cf-46a8-850b-86aa7c9bc195 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:49:16.929422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:49:16.930548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:49:16.930835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:49:16.931108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:49:16.932874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:49:16.955027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:49:16.961153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:49:16.994033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1156/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:49:25.769054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:49:26.981121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:50:19.317123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:50:19.318588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:50:19.334119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6049b1-ce61-47e4-ac9f-40f9e57b5ca4 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:50:19.339135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:50:19.339730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:50:19.340014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:50:19.340290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:50:19.348710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:50:19.350329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:50:19.360715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1148/2304] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 15:50:29.355942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:52:15.995578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:52:16.005698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823794f7-8841-42ae-ac1e-011831bcec32 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:52:16.009257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:52:16.009881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:52:16.010161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:52:16.010433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:52:16.018635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:52:16.020057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:52:16.028730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1157/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:52:18.598720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:52:18.607944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec843e75-0047-44c2-b4da-f4e374ea0f92 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:52:18.611473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:52:18.612060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:52:18.612338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:52:18.612607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:52:18.620857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:52:18.622448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:52:18.631728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1149/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:52:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:52:22.679911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:52:22.733902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d75e5a-b1f0-4d80-8427-aef3c82062a4 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:52:22.738052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:52:22.738640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:52:22.738920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:52:22.739188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:52:22.747628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:52:22.749064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:52:22.758913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1158/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:52:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:52:28.627634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:52:32.335444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:52:32.389493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51114ad1-5e9e-42a8-9b38-66fa3201a149 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:52:32.393144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:52:32.393788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:52:32.394058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:52:32.394334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:52:32.449843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:52:32.451276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:52:32.452321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:52:32.475939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1150/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:52:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:52:32.754771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:52:42.471640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=53 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:52:50.204780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:52:50.206455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:52:50.273333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff7d1-c4bd-4a2b-a6f5-ab3f07d99138 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:52:50.276929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:52:50.277536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:52:50.277856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:52:50.278136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:52:50.279962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:52:50.360635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:52:50.362120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:52:50.374023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1159/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:52:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:53:00.369180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=59 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:53:22.157423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:53:22.159133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:53:22.174362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1820c4-452f-4306-9684-44034f969639 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:53:22.179745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:53:22.180337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:53:22.180618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:53:22.180893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:53:22.183158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:53:22.259328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:53:22.261672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:53:22.270885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1151/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:53:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:53:23.009783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:53:23.018455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35b572-4206-413a-aa8f-43ba78943aaa service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:53:23.022080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:53:23.022679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:53:23.022963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:53:23.023237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:53:23.025551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:53:23.034798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:53:23.036203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:53:23.044656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1160/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:53:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:53:32.266529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=33 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:53:33.040453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:54:16.193264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:54:16.202805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8df89-5bc8-45f4-aa22-8dd8ea45256b service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:54:16.207095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:54:16.207699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:54:16.207981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:54:16.208253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:54:16.210123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:54:16.224066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:54:16.225511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:54:16.237099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1152/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:54:26.232144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:55:24.307158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:55:24.403163 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d582cd-00b7-4710-a70b-389d1d5cfa32 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:55:24.408186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:55:24.408786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:55:24.409071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:55:24.409349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:55:24.411399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:55:24.435844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:55:24.437443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:55:24.447373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1161/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:55:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:55:34.442960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=48 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:57:16.555676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:57:16.624516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12ec82-3fce-45cc-bd23-0dbb937887f3 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:57:16.630403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:57:16.630894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:57:16.631183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:57:16.631461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:57:16.633667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:57:16.722271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:57:16.723877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:57:16.742568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1153/2314] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:57:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:57:22.939101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:57:22.940749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:57:23.002747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1850a-9a5e-47a2-9252-3fec1913ca6e service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:57:23.008510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:57:23.009215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:57:23.009570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:57:23.009951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:57:23.011898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:57:23.094309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:57:23.095946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:57:23.115825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1162/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:57:26.739974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=69 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:57:32.640735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:57:32.652507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa2be2-e5ea-41be-8486-64d0802e172e service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:57:32.658119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:57:32.659369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:57:32.659651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:57:32.659923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:57:32.667937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:57:32.669587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:57:32.697024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1154/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:57:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:57:33.112944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=61 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:57:40.389049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:57:40.410176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d7ba0-a224-429f-a6e5-798a3cdb5099 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:57:40.414093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:57:40.414731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:57:40.415084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:57:40.415428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:57:40.417534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:57:40.535921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:57:40.537392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:57:40.556931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1163/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:57:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:57:42.693833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=13 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:57:50.167632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:57:50.177375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c3d50-a310-416d-83fb-535e5f384187 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:57:50.180962 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:57:50.181565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:57:50.181879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:57:50.182160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:57:50.192795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:57:50.194289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:57:50.207020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1155/2318] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:57:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:57:50.554249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=45 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:58:00.200671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:58:22.554606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:58:22.568565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3fbf9-7eed-49ad-a43f-816145565cf1 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:58:22.574240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:58:22.574843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:58:22.575130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:58:22.575416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:58:22.583365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:58:22.584961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:58:22.614508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1164/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:58:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:58:28.005835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:58:28.025511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c062d94-e4b9-4793-baf7-92ea12faa294 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:58:28.029252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:58:28.029966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:58:28.030255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:58:28.030532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:58:28.038556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:58:28.040001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe427f-7755-4ad4-89ee-3677544e99ad service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:58:28.048156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1156/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:58:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:58:32.611134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=13 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:58:38.044462 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=10 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:59:16.829059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:59:16.838420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207130e-ef4d-4b92-a1b2-44cf06944368 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:59:16.842030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:59:16.842620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:16.842903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:16.843179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:16.854113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:59:16.855553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:59:16.869486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1165/2321] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:59:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:59:26.863827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 15:59:43.335924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:43.336442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:43.336551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:43.364969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] MFA Rules not processed for user `930cbe41fc0741cfa5fbd81645181418`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 15:59:43.366247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:59:43.373322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1157/2322] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 15:59:43.671962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:59:43.680740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7cb5-5d38-4490-be8b-f4f62becb6f5 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:59:43.684700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:59:43.685282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:43.685564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:43.685881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:43.688167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:59:43.697859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:59:43.699262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:59:43.707731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1166/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:59:44.026925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:59:44.038706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734f48-50ff-47a1-8b04-86cfe384b69a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:59:44.044001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:59:44.044613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:44.044893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:44.045165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:44.047570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:59:44.059791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:59:44.061326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:59:44.072435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1158/2324] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:59:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:59:47.149590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:59:47.163812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1de26-760f-4cb9-9129-74163fa2b2b2 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:59:47.170510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:59:47.171117 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:47.171418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:47.171683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:47.173934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:59:47.187376 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:59:47.189328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:59:47.200606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1167/2325] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:59:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:59:50.829499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 15:59:50.843524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b60a0e-92cd-4513-882c-fa539e738bdc service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 15:59:50.859715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 15:59:50.860638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 15:59:50.861168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 15:59:50.861626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 15:59:50.863473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 15:59:50.876861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 15:59:50.878517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 15:59:50.890489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1159/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 15:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 15:59:57.195823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=12 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:00:00.884519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=18 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:00:28.459895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:00:28.461478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:00:28.475793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce9db6-7413-42eb-a54d-74c19d9aae0a service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:00:28.480796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:00:28.481391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:00:28.481676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:00:28.482044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:00:28.490910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:00:28.492483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f625-dbae-4e21-9326-45f5a6c14193 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:00:28.502494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1168/2327] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:00:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:00:38.498001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:01:32.577597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:01:32.592772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a16984-9933-4ae1-8dbe-ca5da8a87f73 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:01:32.599596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:01:32.600296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:01:32.600823 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:01:32.601168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:01:32.603232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:01:32.620313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:01:32.622071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:01:32.642082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1160/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:01:42.634767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:01:44.092037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:01:44.092570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:01:44.092686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:01:44.122380 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] MFA Rules not processed for user `1ca7bef620284345a977088f9a0d2a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:01:44.123712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:01:44.131134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1169/2329] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:01:44.436917 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:01:44.446110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd12e0-d4f0-4adc-a202-4217e409aa0a service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:01:44.449937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:01:44.450523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:01:44.450822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:01:44.451113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:01:44.452817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:01:44.462102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:01:44.463515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:01:44.471836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1161/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:01:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:01:54.120081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:01:54.467759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:02:30.429571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:02:30.443871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a9186-54ef-4e79-af39-ad023151b8ce None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:02:30.449722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:02:30.450328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:02:30.450608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:02:30.450881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:02:30.459560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:02:30.461312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf667a-3093-4157-b37d-b8096555672e service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:02:30.470398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1170/2331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:02:40.466147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:03:31.928764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:31.931200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:32.007375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0244-695f-4541-8b35-c33f99f94b13 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:32.010977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:32.011578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:32.011863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:32.012134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:32.013980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:32.082538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:32.083967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:32.094568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1162/2332] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:36.440547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:36.441061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:36.441197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:36.461723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] MFA Rules not processed for user `1ca7bef620284345a977088f9a0d2a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:03:36.462955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:36.470454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1171/2333] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:03:39.130716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:39.139781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92751140-b184-46cb-915b-d5a85c71ed1c service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:39.143851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:39.144480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:39.144813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:39.145091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:39.147073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:39.157472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:39.159150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:39.168070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1163/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:41.852180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:41.854028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:41.926294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563ff3-cfb2-48c7-9687-13ca9affcc76 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:41.929833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:41.930458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:41.930741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:41.931015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:41.932911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:42.012401 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:42.013883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:42.023398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1172/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:42.068706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:42.078218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4473917-b9f3-4d20-ad35-b3e514349ba5 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:42.081891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:42.082495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:42.082779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:42.083058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:42.091696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:42.093239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:42.102833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1164/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:42.147296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:42.155723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586784f-caf4-4c62-bbf4-d9f53a611499 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:42.159413 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:42.160018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:42.160301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:42.160579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:42.169410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:42.171078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:42.181199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1173/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:42.228377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:03:42.237443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bb90a-3543-4227-82e0-5d89f386eec7 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:03:42.241094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:03:42.241686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:03:42.241998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:03:42.242275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:03:42.244108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:03:42.303688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:03:42.314683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:03:42.316151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:03:42.325270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1165/2338] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:03:52.176811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=67 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:03:52.320862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:25.126931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:25.136080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35b17-2cfa-4463-a54e-f9c651d9e152 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:25.139687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:25.140274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:25.140558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:25.140829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:25.142658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:25.156539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:25.157989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:25.169329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1174/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:05:26.267395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:26.280525 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d535281-a86c-4690-ab2a-695fb68faf50 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:26.287077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:26.287791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:26.288145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:26.288461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:26.299694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:26.301756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:26.316149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1166/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:05:33.288153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:33.289787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:33.379513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b440f8-6118-4cb0-94a6-788f952ee724 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:33.384637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:33.385242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:33.385534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:33.385844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:33.387674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:33.398933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:33.400569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:33.411282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1175/2341] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:05:36.311444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:43.074599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:43.083793 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078cc73-6add-46d0-a6be-6c5a6af5f4a7 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:43.088011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:43.088011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:43.088286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:43.088563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:43.090834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:43.175790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:43.177165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:43.196173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1167/2342] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:05:43.406922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=47 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:46.244961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:46.268856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6efc-1c15-480d-a703-eb86639913c3 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:46.274195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:46.274879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.275237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.275584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.277968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:46.290409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:46.293360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:46.307087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:46.308572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:46.318388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1176/2343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:05:46.325005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7120-639d-44e8-8f6a-08e0d8c03f61 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:46.331445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:46.332098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.332387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.332665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] PATH_INFO: `/v3/users/5d1ab08e56ca4ad692869347a8cd7e6b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.333838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1ab08e56ca4ad692869347a8cd7e6b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:46.335798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:46.387314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1ab08e56ca4ad692869347a8cd7e6b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.387971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1ab08e56ca4ad692869347a8cd7e6b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.388578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1ab08e56ca4ad692869347a8cd7e6b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.395385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1ab08e56ca4ad692869347a8cd7e6b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.407888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9f3dd114-6344-4446-ba8e-7c951a3a5d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1ab08e56ca4ad692869347a8cd7e6b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.420742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1168/2344] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:05:46 2022] DELETE /identity/v3/users/5d1ab08e56ca4ad692869347a8cd7e6b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:05:46.595884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:46.609281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e40f52-8a10-4e20-b899-02e5841f37c3 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:46.666350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:46.666912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.667197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.667972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] PATH_INFO: `/v3/projects/156e628f69d7448dbd0cb45c6dd3ac54` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.669074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156e628f69d7448dbd0cb45c6dd3ac54)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:46.671014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:46.717640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156e628f69d7448dbd0cb45c6dd3ac54', 'request_id': 'req-2fd84038-350d-401f-a13b-4b940e524d78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.724773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156e628f69d7448dbd0cb45c6dd3ac54', 'request_id': 'req-2fd84038-350d-401f-a13b-4b940e524d78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.731613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156e628f69d7448dbd0cb45c6dd3ac54', 'request_id': 'req-2fd84038-350d-401f-a13b-4b940e524d78'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.741732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] The token cache is being invalidate because project 156e628f69d7448dbd0cb45c6dd3ac54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:05:46.742030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.742655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2fd84038-350d-401f-a13b-4b940e524d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:05:46.744127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1177/2345] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:05:46 2022] DELETE /identity/v3/projects/156e628f69d7448dbd0cb45c6dd3ac54 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:05:46.750451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.750744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.751031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.813496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:46.817001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:05:46.819658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:46.883860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1169/2346] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:05:46.893215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619081e4-c105-44bb-969a-322178c1db6e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.893511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619081e4-c105-44bb-969a-322178c1db6e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.893824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619081e4-c105-44bb-969a-322178c1db6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.914079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-619081e4-c105-44bb-969a-322178c1db6e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:05:46.915296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-619081e4-c105-44bb-969a-322178c1db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:46.924043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1178/2347] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:05:46.928198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:46.938150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef7fb-f73b-4c7b-aff9-6861bc1b9fb3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:46.942821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:46.943426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.943712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.943988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.944637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:05:46.944951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:46.946591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:46.958715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1170/2348] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:05:46.962980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619081e4-c105-44bb-969a-322178c1db6e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:46.972611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619081e4-c105-44bb-969a-322178c1db6e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:46.977194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:46.977813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:46.978097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:46.978374 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:46.979009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:46.981036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.017141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1179/2349] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:05:46 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:05:47.021327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:47.031533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884dc0-59ef-44bf-822f-17a607a1aaec admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:47.086411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:47.087006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.087286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.087558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.088183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:47.090156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.110713 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1171/2350] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:05:47 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:05:47.114837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:47.127525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed88c8-8e69-42c5-b7a6-1b04adf6cc3a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:47.132343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:47.132939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.133223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.133500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.134188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:47.135825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.146550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1180/2351] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:05:47.150566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:47.160616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5981ea-d938-445b-a6d1-f37e8f10e67b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:47.165257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:47.165899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.166187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.166466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.167114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:47.168723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a478e7e-b110-4d19-aabf-87655ec99be2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.179231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1172/2352] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:05:47.183622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:47.193346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820005e-12ba-46d0-9315-04c0795f9838 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:47.197963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:47.199384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.199687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.199964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] PATH_INFO: `/v3/projects/5fc8b81f44934eb3b79447f2021448f5/users/e8c0a78c8b0946398b85ad241333ccd6/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.215102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc8b81f44934eb3b79447f2021448f5, user_id=e8c0a78c8b0946398b85ad241333ccd6, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:47.217793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.230831 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1181/2353] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:05:47 2022] PUT /identity/v3/projects/5fc8b81f44934eb3b79447f2021448f5/users/e8c0a78c8b0946398b85ad241333ccd6/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:05:47.241288 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.241577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.242302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.270685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] MFA Rules not processed for user `e8c0a78c8b0946398b85ad241333ccd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:05:47.271877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:47.334957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1173/2354] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:05:47.345870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:05:47.347662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:05:47.354227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:47.428335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d96070-b8fb-47d6-9790-37409b15c166 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:05:47.434779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:05:47.435329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:05:47.435610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:05:47.435881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:05:47.443576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:05:47.445151 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fa25d-c5d7-486b-ba43-83ca7b5eff7c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:05:47.454008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1182/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:05:57.332038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:05:57.450428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:06:47.406803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:47.468489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7901e-2d5a-4d59-b27d-1d65ef8064f3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:47.473004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:47.473630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:47.473941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:47.474219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] PATH_INFO: `/v3/users/e8c0a78c8b0946398b85ad241333ccd6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:47.475326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c0a78c8b0946398b85ad241333ccd6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:47.477287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:47.521769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c0a78c8b0946398b85ad241333ccd6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:47.522429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c0a78c8b0946398b85ad241333ccd6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:47.523042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c0a78c8b0946398b85ad241333ccd6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:47.530417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c0a78c8b0946398b85ad241333ccd6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:47.546149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c0a78c8b0946398b85ad241333ccd6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:47.559579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1174/2356] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:06:47 2022] DELETE /identity/v3/users/e8c0a78c8b0946398b85ad241333ccd6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:06:47.564055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:47.564551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:47.564671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:47.584874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:06:47.586098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:47.641750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1183/2357] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:06:47.651822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:47.653307 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:47.710964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a5d0-44c7-4667-a9a7-3b5389c7938a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:47.714699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:47.715294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:47.715575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:47.715850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:47.724575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:47.726032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abab5a0-2bfb-4395-8fd8-8cecb5603fd2 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:47.735289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1175/2358] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:06:48.206871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.208613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:48.221308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293c97c-8a2d-43ea-abee-211e87fb8714 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.226431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.227056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.227363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.227631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] PATH_INFO: `/v3/projects/5fc8b81f44934eb3b79447f2021448f5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.228756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc8b81f44934eb3b79447f2021448f5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.231586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.278490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc8b81f44934eb3b79447f2021448f5', 'request_id': 'req-f3583628-277a-44fe-8a0d-db6a01066858'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:48.285768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc8b81f44934eb3b79447f2021448f5', 'request_id': 'req-f3583628-277a-44fe-8a0d-db6a01066858'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:48.291886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc8b81f44934eb3b79447f2021448f5', 'request_id': 'req-f3583628-277a-44fe-8a0d-db6a01066858'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:48.302671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] The token cache is being invalidate because project 5fc8b81f44934eb3b79447f2021448f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:06:48.302960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:48.303586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f3583628-277a-44fe-8a0d-db6a01066858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:06:48.305073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1184/2359] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:06:48 2022] DELETE /identity/v3/projects/5fc8b81f44934eb3b79447f2021448f5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:06:48.310067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.310353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.310627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.330025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:06:48.331215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:48.347875 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:06:48.393598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1176/2360] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:06:48.401525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.401859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.402653 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.422793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:06:48.423998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:48.432744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1185/2361] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:06:48.436967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.446520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241de1b-65a9-48a9-8ba5-54e1bdbfdbcf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.451423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.452023 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.452304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.452576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.453219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:06:48.453520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.455151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.467083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1177/2362] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:06:48.471339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.481246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab079-a919-419f-9b58-bbd6a0d9153f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.485979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.486568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.486854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.487131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.487773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.489663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.517752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1186/2363] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:06:48 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:06:48.521430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.531744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984b08e-3d2b-4158-9a20-d7761f54835d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.591225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.591777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.592060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.592332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.593030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.595036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.615913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1178/2364] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:06:48 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:06:48.620052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.630418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b8f6c-1015-401d-b239-abddc98635bc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.635030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.635619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.635901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.636173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.637345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.638964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.648632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1187/2365] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:06:48.652885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.665392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d177ac-37cf-4868-b9ce-3e0c22e391fe admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.670480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.671079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.671361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.671636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.672475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.674149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce757d0-9dca-414a-a51b-61f703d7c5e3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.685338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1179/2366] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:06:48.689810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:06:48.700452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f828-b861-4bc9-901f-8dc90cf94f20 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:06:48.705319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:06:48.706000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.706287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.706572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] PATH_INFO: `/v3/projects/60b5887335144412b87cfb110a25e9ad/users/245563ddf08148b4861527bca4f34ef9/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.721536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b5887335144412b87cfb110a25e9ad, user_id=245563ddf08148b4861527bca4f34ef9, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:06:48.724277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:06:48.737669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1188/2367] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:06:48 2022] PUT /identity/v3/projects/60b5887335144412b87cfb110a25e9ad/users/245563ddf08148b4861527bca4f34ef9/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:06:48.749265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:06:48.749552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:06:48.749863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:06:48.778230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] MFA Rules not processed for user `245563ddf08148b4861527bca4f34ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:06:48.779445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:06:48.844264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1180/2368] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:06:58.733998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:06:58.841342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:42.922460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:42.924171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:43.016223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838462ed-4fba-4f9b-8a19-479a72499863 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:43.019834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:43.020433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:43.020714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:43.020987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:43.022703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:43.118242 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:43.119696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:43.139375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1189/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:07:43.470653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:43.493347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84415ff7-705b-42eb-b328-8cd87b36c8f9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:43.497735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:43.499043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:43.499335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:43.499628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:43.514817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:43.516918 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:43.541973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1181/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:07:43.589639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:43.591283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:43.640112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:43.658353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa3934-535b-4504-8f55-ad265a44d4e7 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:43.662588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:43.663251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:43.663535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:43.663830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:43.665571 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:43.744917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:43.746545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:43.757790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1190/2371] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:07:46.787809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:46.789336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:46.850492 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:46.852016 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:46.869804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ee713-e495-46b1-b3bf-b5b464d67024 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:46.871554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226a5b0-41f4-4d65-99a0-1c50682286aa service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:46.875948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:46.876561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:46.876817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:46.877215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:46.877674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:46.878112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:46.878410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:46.878714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] PATH_INFO: `/v3/users/930cbe41fc0741cfa5fbd81645181418` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:46.880085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:46.895141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:46.895352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=930cbe41fc0741cfa5fbd81645181418)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:46.896688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:46.897281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:46.907732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1182/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:07:46.946383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '930cbe41fc0741cfa5fbd81645181418'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:46.947034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '930cbe41fc0741cfa5fbd81645181418'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:46.947636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '930cbe41fc0741cfa5fbd81645181418'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:46.954867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '930cbe41fc0741cfa5fbd81645181418'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:46.966500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-07db48b6-181b-4775-b199-7714f9dfb561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '930cbe41fc0741cfa5fbd81645181418'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:46.979588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1191/2373] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:07:46 2022] DELETE /identity/v3/users/930cbe41fc0741cfa5fbd81645181418 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:07:47.148553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.162832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f44a4-2ad9-46ce-a091-d5a673efbc7d service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.221412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.222084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.222369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.222647 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] PATH_INFO: `/v3/projects/802e0a1af12748448cbcfecaced16aa6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.233681 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=802e0a1af12748448cbcfecaced16aa6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.235662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.280810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802e0a1af12748448cbcfecaced16aa6', 'request_id': 'req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:47.287312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802e0a1af12748448cbcfecaced16aa6', 'request_id': 'req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:47.292805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802e0a1af12748448cbcfecaced16aa6', 'request_id': 'req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:47.295842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:47.307047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] The token cache is being invalidate because project 802e0a1af12748448cbcfecaced16aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:07:47.307339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:47.307969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e422fbb0-3409-4f2d-9bbf-77dae4607b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:07:47.309574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1183/2374] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:07:47 2022] DELETE /identity/v3/projects/802e0a1af12748448cbcfecaced16aa6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:07:47.316114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.316407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.316686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.336706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:07:47.337948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:47.372067 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:47.398523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1192/2375] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:07:47.461644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.462110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.462395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.482727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:07:47.483946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:47.491790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1184/2376] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:07:47.496008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.505539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822538c-75d1-4281-b4cd-03868b512497 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.510070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.510660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.510937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.511207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.511841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:07:47.512144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.514311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.525494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1193/2377] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:07:47.530099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.539255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa43f7-3db7-4f8b-989c-8efb0401ed77 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.543891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.544481 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.544767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.545046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.545678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.547639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.576949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1185/2378] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:07:47.581359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.591235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74c0b-a603-46d2-b0ae-bdbaaaf4e005 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.646654 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.647174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.647453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.647721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.648366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.650292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.671501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1194/2379] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:07:47.675594 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.685730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afae36c-fd1d-410f-b04c-a4e4239cbdf9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.690294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.690885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.691169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.691447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.692091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.693739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.704241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1186/2380] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:07:47.707919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.718604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb265b1-5f02-4ca8-87a7-dda199f01719 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.724054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.724524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.724806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.725086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.725768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.727381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8bde0-0700-473e-b09a-413295209c63 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.738809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1195/2381] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:07:47.742935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.753016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11feff-275a-40e4-9481-d43940e7725e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.757833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.758872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.758872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.759192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] PATH_INFO: `/v3/projects/74c280a7557d42fd976d4ba767fcd228/users/84ea63e13e5c4595b277e560a8b47b09/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.776869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c280a7557d42fd976d4ba767fcd228, user_id=84ea63e13e5c4595b277e560a8b47b09, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.780819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:47.794562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1187/2382] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:07:47 2022] PUT /identity/v3/projects/74c280a7557d42fd976d4ba767fcd228/users/84ea63e13e5c4595b277e560a8b47b09/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:07:47.804538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.804912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.805285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.836481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] MFA Rules not processed for user `84ea63e13e5c4595b277e560a8b47b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:07:47.837911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:47.900972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1196/2383] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:07:47.905362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:47.966499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232422b-db2c-40fc-b930-825ef6492d41 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:47.971167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:47.971767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:47.972054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:47.972329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:47.972949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:47.974890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.000863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1188/2384] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:07:47 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:07:48.004922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:48.016517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc1e1e-7727-46d9-bd95-78548c3fd6ff None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:48.048398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:48.074896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:48.075518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.075799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.076106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.077230 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:48.079176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.099454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1197/2385] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:07:48 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:07:48.103332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:48.113364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ae00-6b71-44cb-8464-39e0d402c660 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:48.118024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:48.118612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.118898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.119172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.119830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:48.121392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.132434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1189/2386] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:07:48.136479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:48.146943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606185b-2268-4024-83be-70a773181c11 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:48.151879 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:48.152481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.152797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.153092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] PATH_INFO: `/v3/projects/3cb2edb92459450ea5c9eec8647a046c/users/829bff4aaccf419d9a5a47a8e9d957b3/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.167550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb2edb92459450ea5c9eec8647a046c, user_id=829bff4aaccf419d9a5a47a8e9d957b3, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:48.170226 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.182639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1198/2387] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:07:48 2022] PUT /identity/v3/projects/3cb2edb92459450ea5c9eec8647a046c/users/829bff4aaccf419d9a5a47a8e9d957b3/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:07:48.186954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:48.196926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ef25-7760-4451-94da-71107f55bf4f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:48.253156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:48.253841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.254128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.254403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.255037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:48.256617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902adc87-04ba-4427-91fe-272f00dc129d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.261526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:48.270519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1190/2388] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:07:48.271646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:07:48.281844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f882340-b5ff-4166-b18e-15722cd0ace5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:07:48.286502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:07:48.287140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.287422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.287694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] PATH_INFO: `/v3/projects/3cb2edb92459450ea5c9eec8647a046c/users/829bff4aaccf419d9a5a47a8e9d957b3/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.289676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb2edb92459450ea5c9eec8647a046c, user_id=829bff4aaccf419d9a5a47a8e9d957b3, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:07:48.292477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:07:48.303127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1199/2389] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:07:48 2022] PUT /identity/v3/projects/3cb2edb92459450ea5c9eec8647a046c/users/829bff4aaccf419d9a5a47a8e9d957b3/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:07:48.313627 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:07:48.313947 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:07:48.314229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:07:48.341814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] MFA Rules not processed for user `829bff4aaccf419d9a5a47a8e9d957b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:07:48.343029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:07:48.411498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1191/2390] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:07:58.299464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=28,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:07:58.408515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=34 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:38.793081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:38.794971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:38.862871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6c3c-77f3-433c-ac44-15f188550589 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:38.866516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:38.867114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:38.867393 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:38.867666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:38.869371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:38.878507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:38.879907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466fc6d-42d2-4aeb-88d7-8cbe4f72e55b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:38.887646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1200/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:08:48.478113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:48.539967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26508efe-36e9-4543-a42e-a33144b05a07 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:48.544935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:48.545566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:48.545892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:48.546171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] PATH_INFO: `/v3/users/84ea63e13e5c4595b277e560a8b47b09` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:48.547283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ea63e13e5c4595b277e560a8b47b09)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:48.549205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:48.595630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ea63e13e5c4595b277e560a8b47b09'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:48.596272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ea63e13e5c4595b277e560a8b47b09'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:48.596872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ea63e13e5c4595b277e560a8b47b09'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:48.604019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ea63e13e5c4595b277e560a8b47b09'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:48.615925 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ea63e13e5c4595b277e560a8b47b09'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:48.628286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1192/2392] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:08:48 2022] DELETE /identity/v3/users/84ea63e13e5c4595b277e560a8b47b09 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:48.633332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:48.633848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:48.633991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:48.654480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:48.655690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:48.713996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1201/2393] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:08:48.723753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:48.725171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:48.781521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb4a97-fd52-4b92-93ac-cbfff7711fa4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:48.785311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:48.785878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:48.786164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:48.786441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:48.795423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:48.796856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:48.806919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1193/2394] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:08:49.292308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:49.293774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:49.306020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d4469-4e78-4a9b-bcf6-bc64841d9522 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:49.310539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:49.312530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:49.312755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:49.313000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] PATH_INFO: `/v3/projects/74c280a7557d42fd976d4ba767fcd228` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:49.314164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c280a7557d42fd976d4ba767fcd228)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:49.317383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:49.369144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c280a7557d42fd976d4ba767fcd228', 'request_id': 'req-7a791843-7507-4b5b-8926-d39b039873e5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.376479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c280a7557d42fd976d4ba767fcd228', 'request_id': 'req-7a791843-7507-4b5b-8926-d39b039873e5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.382848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c280a7557d42fd976d4ba767fcd228', 'request_id': 'req-7a791843-7507-4b5b-8926-d39b039873e5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.393137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] The token cache is being invalidate because project 74c280a7557d42fd976d4ba767fcd228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:08:49.393424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.394074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.395562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1202/2395] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:08:49 2022] DELETE /identity/v3/projects/74c280a7557d42fd976d4ba767fcd228 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:49.399604 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:49.409274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e834c-5b32-4e09-a6dc-aed1c361e6b3 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:49.428536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:49.467998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:49.468636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:49.468946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:49.469253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] PATH_INFO: `/v3/users/829bff4aaccf419d9a5a47a8e9d957b3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:49.470449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=829bff4aaccf419d9a5a47a8e9d957b3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:49.473082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:49.517176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829bff4aaccf419d9a5a47a8e9d957b3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.517848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829bff4aaccf419d9a5a47a8e9d957b3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.518445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829bff4aaccf419d9a5a47a8e9d957b3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.525247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829bff4aaccf419d9a5a47a8e9d957b3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.537527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f9a293d9-43b7-4559-8be4-521d29c45b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829bff4aaccf419d9a5a47a8e9d957b3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.550883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1194/2396] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:08:49 2022] DELETE /identity/v3/users/829bff4aaccf419d9a5a47a8e9d957b3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:49.811338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:49.812798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:49.833303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a791843-7507-4b5b-8926-d39b039873e5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:49.847880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:49.892921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:49.893546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:49.894523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:49.894802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] PATH_INFO: `/v3/projects/3cb2edb92459450ea5c9eec8647a046c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:49.895901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb2edb92459450ea5c9eec8647a046c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:49.898124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:49.944462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb2edb92459450ea5c9eec8647a046c', 'request_id': 'req-01944bbc-5d1a-48c5-9524-86e2f185901c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.952494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb2edb92459450ea5c9eec8647a046c', 'request_id': 'req-01944bbc-5d1a-48c5-9524-86e2f185901c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.959186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb2edb92459450ea5c9eec8647a046c', 'request_id': 'req-01944bbc-5d1a-48c5-9524-86e2f185901c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.969809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] The token cache is being invalidate because project 3cb2edb92459450ea5c9eec8647a046c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:08:49.970099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.970826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-01944bbc-5d1a-48c5-9524-86e2f185901c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:08:49.972326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1203/2397] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:08:49 2022] DELETE /identity/v3/projects/3cb2edb92459450ea5c9eec8647a046c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:49.978138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:49.978426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:49.978703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:49.999280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:50.000495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:50.058074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1195/2398] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:08:50.067392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.067685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.067963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.088018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:50.089221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:50.096958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1204/2399] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:08:50.101042 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.111419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9130f-3d26-45e7-99a0-f72047f36403 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.116030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.116628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.116908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.117181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.117862 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:08:50.118173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.119764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.131724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1196/2400] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:08:50.135910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.145579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff218e-5173-4c61-84f9-b3690a4f848d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.150464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.151055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.151336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.151607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.152238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.154172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.180219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1205/2401] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:08:50.184318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.194836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c1498-af48-4f15-acbf-44ca65d8116b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.221191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:50.252413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.252997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.253279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.254151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.254776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.256724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.277376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1197/2402] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:08:50.281491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.291592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794239-20cd-401c-8329-c3a169b69807 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.296127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.296716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.296999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.297273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.297931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.299636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.309814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1206/2403] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:08:50.313345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.323684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443fa6-e223-4644-a3da-4cbddc737c35 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.328584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.329174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.329457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.329766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.330409 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.332022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb220cd-3315-4579-96f4-f16b0298a81b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.342811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1198/2404] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:08:50.347020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.356834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb78d5-3a0e-461d-a230-9b5b5950dfa1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.361495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.362178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.362464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.363276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] PATH_INFO: `/v3/projects/78d16b0562e14846af34286a8604da77/users/fab6286210b64418b4eadd4be14fd180/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.378049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d16b0562e14846af34286a8604da77, user_id=fab6286210b64418b4eadd4be14fd180, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.381363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.393583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1207/2405] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:08:50 2022] PUT /identity/v3/projects/78d16b0562e14846af34286a8604da77/users/fab6286210b64418b4eadd4be14fd180/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:50.403423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.403709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.403982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.431305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] MFA Rules not processed for user `fab6286210b64418b4eadd4be14fd180`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:50.432500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:50.496889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1199/2406] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:08:50.501348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.546788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:50.564311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264b245-9f2f-48ce-883d-ff7880f68b74 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.569615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.570256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.570541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.570816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.571449 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.573344 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.598680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1208/2407] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:08:50.602747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.614078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500d97f-ecf5-4540-b1bc-bce0a96be957 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.670287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.670891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.671172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.671444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.672075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.674017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.695858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1200/2408] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:08:50.699896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.710474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16fee-c37f-434e-b663-e8df1260efa0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.715054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.715652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.715939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.716219 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.716865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.718588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.729525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1209/2409] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:08:50.733344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.743806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b11b-a242-4f28-b2e1-abaf69b94ddf admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.748425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.749003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.749291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.749570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.750246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.751843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230a842-5039-4075-bb79-f4973cc17265 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.761662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1201/2410] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:08:50.765533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.775132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb67f7b-ebce-4d5d-9060-9eb801058329 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.779667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.780259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.780536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.780807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] PATH_INFO: `/v3/projects/3bb62bb3bb98460ca0507e479c7370ec/users/6d30a6cd939444f689dacd3dbb78a412/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.795277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb62bb3bb98460ca0507e479c7370ec, user_id=6d30a6cd939444f689dacd3dbb78a412, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.797950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:50.810627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1210/2411] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:08:50 2022] PUT /identity/v3/projects/3bb62bb3bb98460ca0507e479c7370ec/users/6d30a6cd939444f689dacd3dbb78a412/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:50.820369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.820662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.820946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.849291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] MFA Rules not processed for user `6d30a6cd939444f689dacd3dbb78a412`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:50.850569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:50.854531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:50.915566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1202/2412] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:08:50.919970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:50.979973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed176-a16c-41c3-a3ff-1790bf24cebb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:50.984455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:50.985048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:50.985327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:50.985599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:50.986257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:50.988839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.013938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1211/2413] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:08:50 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:08:51.018072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:51.028682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf85f4-377b-4685-a5b6-0de7fb404f7e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:51.086736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:51.087344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.087628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.087908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.088559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:51.090538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.112858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1203/2414] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:08:51 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:08:51.116921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:51.127137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba967-6886-45ac-9daa-b398fd780309 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:51.131709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:51.132298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.132582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.132856 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.133486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:51.135162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.146943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1212/2415] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:08:51.150822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:51.161318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a474-b8bc-42f7-8dba-bd08fd5f143b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:51.166135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:51.166799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.167108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.167357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] PATH_INFO: `/v3/projects/1bd79358a1ac4e058b88844233eb9954/users/7e3d44a52c984803b84b06aa21d82576/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.182047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd79358a1ac4e058b88844233eb9954, user_id=7e3d44a52c984803b84b06aa21d82576, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:51.184692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.197334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1204/2416] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:08:51 2022] PUT /identity/v3/projects/1bd79358a1ac4e058b88844233eb9954/users/7e3d44a52c984803b84b06aa21d82576/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:51.201462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:51.211802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddae1d-c35b-4dc0-8786-b425162e3156 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:51.266920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:51.267458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.267751 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.268066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.268698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:51.270888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca04c5-c919-472d-8147-41acd28c0133 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.280494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1213/2417] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:08:51.284641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:08:51.295338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d94cab-f7d5-4941-8731-b255330ce809 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:08:51.299974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:08:51.300645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.300928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.301205 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] PATH_INFO: `/v3/projects/1bd79358a1ac4e058b88844233eb9954/users/7e3d44a52c984803b84b06aa21d82576/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.303138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd79358a1ac4e058b88844233eb9954, user_id=7e3d44a52c984803b84b06aa21d82576, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:08:51.305818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:08:51.316431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1205/2418] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:08:51 2022] PUT /identity/v3/projects/1bd79358a1ac4e058b88844233eb9954/users/7e3d44a52c984803b84b06aa21d82576/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:08:51.327482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:08:51.327776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:08:51.328056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:08:51.336518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:08:51.358215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] MFA Rules not processed for user `7e3d44a52c984803b84b06aa21d82576`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:08:51.359470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:08:51.427957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1214/2419] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:09:01.312629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:09:01.424997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:09:43.565606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:09:43.567393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:43.637018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45627-f031-43f6-be70-7cd5d3c40145 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:09:43.641171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:09:43.641798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:09:43.642085 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:09:43.642359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:09:43.644076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:43.710651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:09:43.712121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:09:43.730306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1206/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:09:43.771846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:09:43.781945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607614e-c84a-4c90-b7f4-d5c65befba09 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:09:43.785419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:09:43.786039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:09:43.786319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:09:43.786593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:09:43.788333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:43.855600 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:09:43.857576 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:09:43.878512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1215/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:09:43.880381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:09:43.881789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:43.893322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4054f2-8c50-4679-9d92-4930ed72ef5b service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:09:43.896957 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:09:43.897561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:09:43.897888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:09:43.898329 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:09:43.899886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:44.001126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:09:44.002631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:09:44.015104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:09:44.021200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:09:44.024788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1207/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:09:44.030650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc07e49-2692-4224-99c4-da8e047701e2 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:09:44.034510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:09:44.035112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:09:44.035397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:09:44.035681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:09:44.037432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] Could not find project: 3cb2edb92459450ea5c9eec8647a046c.: keystone.exception.ProjectNotFound: Could not find project: 3cb2edb92459450ea5c9eec8647a046c. Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 12 16:09:44.050756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 16:09:44.052148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 16:09:44.053657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 16:09:44.055278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 16:09:44.056677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 3cb2edb92459450ea5c9eec8647a046c. Sep 12 16:09:44.056677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 12 16:09:44.056677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1216/2423] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:09:44 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 12 16:09:44.151789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:09:44.161338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d01c6-58f1-43e3-8ff4-ae4edb1ed407 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:09:44.165419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:09:44.166200 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:09:44.166608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:09:44.166905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:09:44.177347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:09:44.178830 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:09:44.188761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1208/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:09:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:09:54.041274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=40 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:09:54.184150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=8 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:10:37.258400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:10:37.260791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:10:37.329220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e265-a456-40f0-b809-fca451c5016a service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:10:37.334806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:10:37.335402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:10:37.335683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:10:37.335956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:10:37.337646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:10:37.348546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:10:37.350147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:10:37.359895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1217/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:10:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:10:47.355610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:37.807049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:37.817407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d18539-9322-4df0-aa8e-3547435dfb59 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:37.826119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:37.826119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:37.826119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:37.826119 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:37.826808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:37.842512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:37.844469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:37.858413 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1209/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:38.292802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:38.303213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82150-d83f-41cc-95f3-0a281bb8ff36 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:38.307022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:38.307633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:38.307916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:38.308218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:38.386027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:38.387760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:38.396732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1218/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:39.387295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:39.388753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:39.402380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b5654-a632-44c3-a8da-f7b7988cbf33 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:39.407936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:39.408540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:39.408828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:39.409099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:39.410843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:39.424679 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:39.426330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:39.438289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1210/2428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:42.586979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:42.588515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:42.646220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445e5ff-4ba4-4e44-8325-0e65ea3ef856 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:42.650434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:42.651040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:42.651324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:42.651601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:42.653348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:42.702929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:42.712924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c72ec-45d4-489e-ac9e-9a77c42587ca service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:42.717054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:42.718097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:42.719000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:42.719553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:42.722777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:42.724422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:42.736089 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1219/2429] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:42.785751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:42.787358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:42.797244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1211/2430] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:11:43.159572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:43.171536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fea07-9cb0-4dab-b316-eb70a6fd20d7 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:43.184362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:43.189053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:43.189291 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:43.190634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:43.193113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:43.206957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:43.208983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:43.235727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1220/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:43.408593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:43.410571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:43.427315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa67447-12da-4996-998c-d99a09d89b99 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:43.433015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:43.433782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:43.434318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:43.434803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] PATH_INFO: `/v3/users/10a5c8ccdc2b49d68a6c088207d593d3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:43.436895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a5c8ccdc2b49d68a6c088207d593d3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:43.441601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:43.511513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a5c8ccdc2b49d68a6c088207d593d3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:43.513761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a5c8ccdc2b49d68a6c088207d593d3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:43.514473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a5c8ccdc2b49d68a6c088207d593d3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:43.526546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a5c8ccdc2b49d68a6c088207d593d3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:43.544500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ef27679-3132-4a71-8e65-2031974f5877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a5c8ccdc2b49d68a6c088207d593d3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:43.560308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1212/2432] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:43 2022] DELETE /identity/v3/users/10a5c8ccdc2b49d68a6c088207d593d3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:44.336451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:44.347618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104798-440c-485d-9c60-f1e963b91b65 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:44.407079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:44.456945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:44.457687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.458185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.458517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] PATH_INFO: `/v3/projects/9b6d13c010304755ba81ce8f31459b87` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.459650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6d13c010304755ba81ce8f31459b87)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:44.461686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:44.521420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6d13c010304755ba81ce8f31459b87', 'request_id': 'req-b656e52e-052d-433e-bf22-a772d03f387c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:44.528947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6d13c010304755ba81ce8f31459b87', 'request_id': 'req-b656e52e-052d-433e-bf22-a772d03f387c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:44.536278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6d13c010304755ba81ce8f31459b87', 'request_id': 'req-b656e52e-052d-433e-bf22-a772d03f387c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:44.548295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] The token cache is being invalidate because project 9b6d13c010304755ba81ce8f31459b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:44.548589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:44.549265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b656e52e-052d-433e-bf22-a772d03f387c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:44.550812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1221/2433] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:44 2022] DELETE /identity/v3/projects/9b6d13c010304755ba81ce8f31459b87 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:44.557689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.558317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.558317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.578914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:44.580169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:44.626646 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:44.651530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1213/2434] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:44.660329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.660626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.660917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.688390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:44.689668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:44.698215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1222/2435] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:44.703550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:44.715706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4f05c-9e6f-496f-953c-704d8fc4414b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:44.721414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:44.722163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.722518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.722889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.723690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:44.723940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:44.726185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:44.743212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1214/2436] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:11:44.744682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:44.754738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee69ee-13ad-4fe6-b7a8-548fe078beca None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:44.760963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:44.761760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.762139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.762713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.763489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:44.766947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:44.832924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1223/2437] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:44 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:44.837245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:44.849635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81dc05-8672-4e43-8b8e-29624baaf725 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:44.940466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:44.941392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:44.941865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:44.942318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:44.943121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:44.946458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:44.979291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1215/2438] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:44 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:44.983948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.008763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304803f-9935-4221-a80b-b8a14cb90307 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.014419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.015047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.015337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.015652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.016266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.018333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.035985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1224/2439] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:45.037718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.055440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30061779-8cfc-4002-829b-780e95463865 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.066436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.067157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.067509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.067888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.068792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.070785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454734-57d9-44a8-8ae8-ffc39c7071c0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.094704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1216/2440] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:45.095575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.108948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefbe34-4479-4745-af09-4e0ccba66810 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.114251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.115198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.115458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.115789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] PATH_INFO: `/v3/projects/de833de2ba294244859ccbe233ced7ad/users/deb44b514dbf425aa108690d647179f4/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.139531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de833de2ba294244859ccbe233ced7ad, user_id=deb44b514dbf425aa108690d647179f4, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.144195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.169748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1225/2441] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:45 2022] PUT /identity/v3/projects/de833de2ba294244859ccbe233ced7ad/users/deb44b514dbf425aa108690d647179f4/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:45.178912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.179480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.179835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.202895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:45.204398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.353074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1217/2442] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:45.360625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.362688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.412761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.414666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.433041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b5820-936c-40cb-b30d-c5fb7994e2cd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.440466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.441229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.441578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.442136 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] PATH_INFO: `/v3/users/fab6286210b64418b4eadd4be14fd180` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.445740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab6286210b64418b4eadd4be14fd180)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.449863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.490102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c1fab-69bb-44a4-9fe6-3abd4df42e12 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.498683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.501260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.502486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.502982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.521382 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.523726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.532067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:45.533806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab6286210b64418b4eadd4be14fd180'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.534582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab6286210b64418b4eadd4be14fd180'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.535621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab6286210b64418b4eadd4be14fd180'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.536736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1226/2443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:45.537681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.538963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.550068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:45.556387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfd72-2545-42f5-b78a-0cffb63363b1 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.568070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] PATH_INFO: `/v3/users/245563ddf08148b4861527bca4f34ef9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.568869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab6286210b64418b4eadd4be14fd180'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.569938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=245563ddf08148b4861527bca4f34ef9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.571756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.604518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab6286210b64418b4eadd4be14fd180'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.624408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1218/2444] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:45 2022] DELETE /identity/v3/users/fab6286210b64418b4eadd4be14fd180 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:45.624408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.626114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.629617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245563ddf08148b4861527bca4f34ef9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.630664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245563ddf08148b4861527bca4f34ef9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.631440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245563ddf08148b4861527bca4f34ef9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.642177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245563ddf08148b4861527bca4f34ef9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.666346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-efe11fef-2e40-415e-8626-bc1919ae7969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245563ddf08148b4861527bca4f34ef9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:45.689943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1227/2445] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:45 2022] DELETE /identity/v3/users/245563ddf08148b4861527bca4f34ef9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:45.693467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965803-8944-4ee2-ae22-c29cb5cf0df1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.693779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965803-8944-4ee2-ae22-c29cb5cf0df1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.694056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965803-8944-4ee2-ae22-c29cb5cf0df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.710771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f3e22-7fd2-4a03-8cfa-49b5d58cddf4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.715434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f9965803-8944-4ee2-ae22-c29cb5cf0df1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:45.716807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f9965803-8944-4ee2-ae22-c29cb5cf0df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.813920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.816480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.816876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.817385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.823113 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.840215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1228/2446] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:45.842088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.842621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.843131 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.874093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:45.875527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.888954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1229/2447] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:45.891641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.893913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:45.918485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f09634-7549-4351-a89a-bc2f5a8f2464 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.926574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.927201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.927587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.927865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.946878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.948573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.964793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1230/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:45.965818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:45.966373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.967919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.975216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf283b-1fa6-4218-8a86-edc685621dcb service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:45.979366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:45.979964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:45.980245 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:45.980577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:45.995110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:45.997153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:45.997315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1219/2449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:46.010438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1231/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:47.327963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:47.329511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.342705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:47.344146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.356958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1307c4-c41f-45a5-95f3-a19b7c0094f3 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:47.367565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:47.368237 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.368603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.368883 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] PATH_INFO: `/v3/projects/60b5887335144412b87cfb110a25e9ad` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.370127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b5887335144412b87cfb110a25e9ad)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:47.372059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:47.381256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48851059-60ec-4038-ac27-c8897b3fd457 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:47.390722 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:47.391367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.391650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.391924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] PATH_INFO: `/v3/projects/78d16b0562e14846af34286a8604da77` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.395340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d16b0562e14846af34286a8604da77)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:47.397619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:47.434872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:47.479673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:47.493873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b5887335144412b87cfb110a25e9ad', 'request_id': 'req-6b3f8406-5fb4-495e-aa43-6bf40864ed01'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.494202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d16b0562e14846af34286a8604da77', 'request_id': 'req-89007e94-61f6-459a-9a46-92a18af6b098'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.512775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b5887335144412b87cfb110a25e9ad', 'request_id': 'req-6b3f8406-5fb4-495e-aa43-6bf40864ed01'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.516164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d16b0562e14846af34286a8604da77', 'request_id': 'req-89007e94-61f6-459a-9a46-92a18af6b098'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.526526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d16b0562e14846af34286a8604da77', 'request_id': 'req-89007e94-61f6-459a-9a46-92a18af6b098'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.537539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b5887335144412b87cfb110a25e9ad', 'request_id': 'req-6b3f8406-5fb4-495e-aa43-6bf40864ed01'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.544379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] The token cache is being invalidate because project 78d16b0562e14846af34286a8604da77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:47.545827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.547096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.555239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1232/2451] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:47 2022] DELETE /identity/v3/projects/78d16b0562e14846af34286a8604da77 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:47.560952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:47.562546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.568463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] The token cache is being invalidate because project 60b5887335144412b87cfb110a25e9ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:47.569574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.571224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6b3f8406-5fb4-495e-aa43-6bf40864ed01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.575692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1220/2452] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:47 2022] DELETE /identity/v3/projects/60b5887335144412b87cfb110a25e9ad => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:47.581512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06b00-e943-49d4-aa06-db61b03577e0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.582071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06b00-e943-49d4-aa06-db61b03577e0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.582737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06b00-e943-49d4-aa06-db61b03577e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.652083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4ec06b00-e943-49d4-aa06-db61b03577e0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:47.656774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4ec06b00-e943-49d4-aa06-db61b03577e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.704721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007e94-61f6-459a-9a46-92a18af6b098 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:47.715748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:47.716703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.717150 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.717625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] PATH_INFO: `/v3/users/6d30a6cd939444f689dacd3dbb78a412` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.719521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d30a6cd939444f689dacd3dbb78a412)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:47.723758 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:47.749921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1221/2453] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:11:47.759149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.759403 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.759812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.792607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:47.794372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.816354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1222/2454] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:47.819180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:47.848912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d30a6cd939444f689dacd3dbb78a412'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.849842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d30a6cd939444f689dacd3dbb78a412'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.851321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d30a6cd939444f689dacd3dbb78a412'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.859500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fa4d9-3af9-467e-800c-6e490a71599c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:47.880512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:47.881468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:47.881468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:47.881755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:47.882362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:47.882677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:47.883548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d30a6cd939444f689dacd3dbb78a412'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.884477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:47.909507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d30a6cd939444f689dacd3dbb78a412'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:47.920407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1223/2455] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:11:47.922518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:47.924885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:47.943273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1233/2456] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:47 2022] DELETE /identity/v3/users/6d30a6cd939444f689dacd3dbb78a412 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:47.962293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c71150-1971-4445-b754-5230bab0080a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.106865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.107683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.108069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.108450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.109297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.111444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:48.153357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1224/2457] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:47 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:11:48.160543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:48.176959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51923f86-9c88-4077-bd02-e26f4fd72258 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.270101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:48.299794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.300977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.301403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.301691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.302875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.324843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:48.361635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1234/2458] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:48 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:48.363389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:48.376581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb40402-bd5d-48c5-a4fc-0312e57e7efc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.382534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.383162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.383449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.383727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.384393 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.386058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:48.398146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1225/2459] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:48.402246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:48.417821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f111-43cd-4766-a02c-1a93f973eb8a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.425005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.426293 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.426592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.426940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.427854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.430197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb56c-8fc8-4173-8aae-a778fc5358d6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:48.448494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1235/2460] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:48.455410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:48.471051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7afe99-1a24-4ec7-b187-23410c871f0c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.485553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.490505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.490829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.491099 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] PATH_INFO: `/v3/projects/aab81c9b225f4991a8f39cae77c40af0/users/c8b9d168291e49469d4e0f1072403395/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.495206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:48.518965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab81c9b225f4991a8f39cae77c40af0, user_id=c8b9d168291e49469d4e0f1072403395, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.524639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:48.547149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1226/2461] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:48 2022] PUT /identity/v3/projects/aab81c9b225f4991a8f39cae77c40af0/users/c8b9d168291e49469d4e0f1072403395/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:48.559916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.560034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.560824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.614964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] MFA Rules not processed for user `c8b9d168291e49469d4e0f1072403395`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:48.616789 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:48.793253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1236/2462] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:48.798849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:48.925276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6abf-4cec-41a6-99df-3d5270f8d51c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:48.933910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:48.937182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:48.938139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:48.938526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:48.940714 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:48.943104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.009408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1227/2463] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:48 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:49.013116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.029065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3310e6-c165-49d8-860a-38bbf08c249a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.141954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.142957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.143339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.145121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.146155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.150627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.204514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1237/2464] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:49 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:11:49.207395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.218258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abe7be-a305-4c89-8618-5a57581d1685 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.223707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.224331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.224607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.224880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.226105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.227803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.246917 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.253250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1228/2465] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:49.261471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c05e3-992b-4f24-8c63-56e393dab5ba admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.268496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.269213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.269491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.269847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] PATH_INFO: `/v3/projects/b2427414b9594c2a86f59c3614b297ce/users/402347a67a704859bcf2c52190a7bc0d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.301517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2427414b9594c2a86f59c3614b297ce, user_id=402347a67a704859bcf2c52190a7bc0d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.306358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.331061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.333198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:49.340389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1238/2466] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:49 2022] PUT /identity/v3/projects/b2427414b9594c2a86f59c3614b297ce/users/402347a67a704859bcf2c52190a7bc0d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:11:49.344438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.352698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007dd1-3232-49dc-aba3-123a42d642a9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.397942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:49.469507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b25d0-e16c-4d76-8945-0f389c3edb2e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.476398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.477897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.478504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.479148 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.480039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.482322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.490935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.490935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.491634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.492661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] PATH_INFO: `/v3/projects/3bb62bb3bb98460ca0507e479c7370ec` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.494568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb62bb3bb98460ca0507e479c7370ec)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.497692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.506252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1239/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:49.507862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.525696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4603e4-b9a9-4513-9ce3-2338e155645a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.535435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.538533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.538834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.539372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] PATH_INFO: `/v3/projects/b2427414b9594c2a86f59c3614b297ce/users/402347a67a704859bcf2c52190a7bc0d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.544166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2427414b9594c2a86f59c3614b297ce, user_id=402347a67a704859bcf2c52190a7bc0d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.548235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb8c4d-3d56-4ca2-9d7f-e58b8ec665c4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.562601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1240/2468] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:49 2022] PUT /identity/v3/projects/b2427414b9594c2a86f59c3614b297ce/users/402347a67a704859bcf2c52190a7bc0d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:49.585298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.585766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.588639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.623287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb62bb3bb98460ca0507e479c7370ec', 'request_id': 'req-9abc1dfa-2fa7-4b36-979a-a67337375a2a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:49.642730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb62bb3bb98460ca0507e479c7370ec', 'request_id': 'req-9abc1dfa-2fa7-4b36-979a-a67337375a2a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:49.654929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:49.659846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb62bb3bb98460ca0507e479c7370ec', 'request_id': 'req-9abc1dfa-2fa7-4b36-979a-a67337375a2a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:49.675606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] MFA Rules not processed for user `402347a67a704859bcf2c52190a7bc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:49.678063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:49.686274 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] The token cache is being invalidate because project 3bb62bb3bb98460ca0507e479c7370ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:49.686576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:49.687665 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:49.696167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1229/2469] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:49 2022] DELETE /identity/v3/projects/3bb62bb3bb98460ca0507e479c7370ec => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:11:49.698494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.702852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:49.803008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1241/2470] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:49.812464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1dfa-2fa7-4b36-979a-a67337375a2a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.816352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:49.817730 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:49.828501 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.830185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.831161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.831961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] PATH_INFO: `/v3/users/7e3d44a52c984803b84b06aa21d82576` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.834212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3d44a52c984803b84b06aa21d82576)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.837972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.912859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd68ebb-095a-4d3e-9b74-935962e0b083 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:49.917819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:49.920567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:49.920871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:49.921147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:49.938591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:49.942204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:49.975587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1242/2471] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:11:50.008005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3d44a52c984803b84b06aa21d82576'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.009066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3d44a52c984803b84b06aa21d82576'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.011539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3d44a52c984803b84b06aa21d82576'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.023909 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3d44a52c984803b84b06aa21d82576'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.033620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:50.035922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.055370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1db1a4-fdbd-4a7a-aeea-57f0f455bdcd service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:50.057535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0983a9cd-ac3d-4323-9d10-d47f1335da2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3d44a52c984803b84b06aa21d82576'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.067601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:50.068327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:50.068894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:50.070168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] PATH_INFO: `/v3/users/c8b9d168291e49469d4e0f1072403395` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:50.071835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b9d168291e49469d4e0f1072403395)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:50.086977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1230/2472] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:49 2022] DELETE /identity/v3/users/7e3d44a52c984803b84b06aa21d82576 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:50.087165 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:50.149007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:50.189965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b9d168291e49469d4e0f1072403395'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.191873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b9d168291e49469d4e0f1072403395'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.194767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b9d168291e49469d4e0f1072403395'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.209655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b9d168291e49469d4e0f1072403395'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.231611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b9d168291e49469d4e0f1072403395'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.267223 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1243/2473] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:50 2022] DELETE /identity/v3/users/c8b9d168291e49469d4e0f1072403395 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:50.269228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:50.269932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:50.270116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:50.296070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:50.297248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.382868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1231/2474] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:50.409175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:50.410798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.522619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377578bc-d248-43f7-9be2-9322ae6f01b2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:50.530779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:50.531702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:50.531996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:50.532791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:50.565575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:50.567209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:50.587982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1244/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:11:50.621431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:50.622985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.638008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:50.660253 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:50.662044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.687944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7e0e1-3a7d-42de-bed5-898c51833bc8 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:50.714993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:50.716599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:50.717139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:50.717661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] PATH_INFO: `/v3/users/1ca7bef620284345a977088f9a0d2a07` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:50.720765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca7bef620284345a977088f9a0d2a07)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:50.728961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:50.791829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abeea3-b385-41eb-9016-a299cef0969c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:50.799619 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:50.800076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:50.800362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:50.800722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:50.803127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:50.824666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:50.827709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:50.828277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca7bef620284345a977088f9a0d2a07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.830481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca7bef620284345a977088f9a0d2a07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.831725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca7bef620284345a977088f9a0d2a07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.845415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1232/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:50.848043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca7bef620284345a977088f9a0d2a07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.896175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca7bef620284345a977088f9a0d2a07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:50.924430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1245/2477] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:50 2022] DELETE /identity/v3/users/1ca7bef620284345a977088f9a0d2a07 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:11:51.452954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:51.454444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:51.483768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb18f7f-eee1-4044-8f22-5cb91c411e0b service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:51.601201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:51.603325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:51.612295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:51.612954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:51.613239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:51.613637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] PATH_INFO: `/v3/projects/1bd79358a1ac4e058b88844233eb9954` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:51.617183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd79358a1ac4e058b88844233eb9954)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:51.619193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:51.642676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc4f61-4a9e-4f61-8678-4d22dde2407a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:51.665302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:51.667814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:51.667814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:51.671548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] PATH_INFO: `/v3/projects/6661c7cb48d947439a324c3f795fd57c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:51.673364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6661c7cb48d947439a324c3f795fd57c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:51.679830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:51.720502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:51.739143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd79358a1ac4e058b88844233eb9954', 'request_id': 'req-88af9297-f139-4ad3-9ea1-2fd7cc38d084'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.750894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd79358a1ac4e058b88844233eb9954', 'request_id': 'req-88af9297-f139-4ad3-9ea1-2fd7cc38d084'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.771870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd79358a1ac4e058b88844233eb9954', 'request_id': 'req-88af9297-f139-4ad3-9ea1-2fd7cc38d084'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.798388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6661c7cb48d947439a324c3f795fd57c', 'request_id': 'req-48e56dbe-07ac-44f4-b5ed-de422f95714c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.801395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] The token cache is being invalidate because project 1bd79358a1ac4e058b88844233eb9954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:51.801903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.804068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-88af9297-f139-4ad3-9ea1-2fd7cc38d084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.813111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1233/2478] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:51 2022] DELETE /identity/v3/projects/1bd79358a1ac4e058b88844233eb9954 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:51.816973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab384-ac48-424b-8c4e-22ce19976328 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:51.818399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab384-ac48-424b-8c4e-22ce19976328 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:51.818802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ab384-ac48-424b-8c4e-22ce19976328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:51.820892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6661c7cb48d947439a324c3f795fd57c', 'request_id': 'req-48e56dbe-07ac-44f4-b5ed-de422f95714c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.831463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6661c7cb48d947439a324c3f795fd57c', 'request_id': 'req-48e56dbe-07ac-44f4-b5ed-de422f95714c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.856660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-630ab384-ac48-424b-8c4e-22ce19976328 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:51.860916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] The token cache is being invalidate because project 6661c7cb48d947439a324c3f795fd57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:51.861323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.862099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-48e56dbe-07ac-44f4-b5ed-de422f95714c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:51.864199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1246/2479] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:51 2022] DELETE /identity/v3/projects/6661c7cb48d947439a324c3f795fd57c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:51.868394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-630ab384-ac48-424b-8c4e-22ce19976328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:51.870172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2680-6a15-4b6d-a35f-fd08e50594c8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:51.870599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2680-6a15-4b6d-a35f-fd08e50594c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:51.870935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef2680-6a15-4b6d-a35f-fd08e50594c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:51.891257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:51.938268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-30ef2680-6a15-4b6d-a35f-fd08e50594c8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:51.939637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-30ef2680-6a15-4b6d-a35f-fd08e50594c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.031468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1234/2480] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:52.037299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.038467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.038893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.073293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:52.075980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.098993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1235/2481] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:52.100768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.101855 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1247/2482] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:52.106911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.107769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.108277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.117160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99f7e-ce22-4bfd-9aec-bfeae6dd0148 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.122193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.122822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.123101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.123374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.124036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:52.124341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.126036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.150571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1236/2483] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:11:52.150571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.153059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.155957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:52.157535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.170737 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1248/2484] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:52.171873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.172868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523552-2b82-48a7-9070-327996971ee8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.181428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.182435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.182983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.183497 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] PATH_INFO: `/v3/projects/aab81c9b225f4991a8f39cae77c40af0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.184898 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab81c9b225f4991a8f39cae77c40af0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.187819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.198631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1991cd-5dd1-4cd7-9676-93f14a601342 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.213655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.214399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.214781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.215745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.217094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.223681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.288221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab81c9b225f4991a8f39cae77c40af0', 'request_id': 'req-5e873978-903e-4f1a-aea8-3e33d8d51d6e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.292170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1249/2485] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:52.297015 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.314503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab81c9b225f4991a8f39cae77c40af0', 'request_id': 'req-5e873978-903e-4f1a-aea8-3e33d8d51d6e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.323334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab81c9b225f4991a8f39cae77c40af0', 'request_id': 'req-5e873978-903e-4f1a-aea8-3e33d8d51d6e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.352031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] The token cache is being invalidate because project aab81c9b225f4991a8f39cae77c40af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:52.352325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.354059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.359932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1237/2486] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:52 2022] DELETE /identity/v3/projects/aab81c9b225f4991a8f39cae77c40af0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:52.359932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.361502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.450346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3776a4-a081-4d46-8d22-c40f654b37e5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.460716 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.461882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.462816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.463196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.464377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:52.464801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.470667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.501380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873978-903e-4f1a-aea8-3e33d8d51d6e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.510014 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.511257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.511761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.512184 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.513024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.515515 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.519237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1250/2487] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:11:52.519237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.525375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.565500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee0fa-b212-4e07-b1c2-2035d3e9edd3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.571625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1238/2488] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:52.571625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.571889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.571996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.572888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.573024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.573194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] PATH_INFO: `/v3/users/402347a67a704859bcf2c52190a7bc0d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.578076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402347a67a704859bcf2c52190a7bc0d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.580753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.583050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:52.587510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adddba-e343-4774-82a9-9117b9d8f911 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.595032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.595676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.596108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.597127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.597696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.599776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1cdce-b2d3-4088-b43e-4e90eaae3942 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.659123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:52.683173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402347a67a704859bcf2c52190a7bc0d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.686670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402347a67a704859bcf2c52190a7bc0d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.686805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1239/2489] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/projects => generated 361 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:52.686916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402347a67a704859bcf2c52190a7bc0d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.687036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.687036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.687305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.695476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402347a67a704859bcf2c52190a7bc0d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.710683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402347a67a704859bcf2c52190a7bc0d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:52.730201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] MFA Rules not processed for user `deb44b514dbf425aa108690d647179f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:52.734182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.742800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1251/2490] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:52 2022] DELETE /identity/v3/users/402347a67a704859bcf2c52190a7bc0d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:52.742800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.746176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.853256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d9d9-4221-426d-a3b2-1bbfc9902c19 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.860166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1240/2491] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:52.864516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.869863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.875277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.876252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.876252 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.876474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.878542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.881743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.904486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1252/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:52.906238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:52.908277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:52.913285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b4bdb-6f60-486e-8fc3-8ec4b11a21a1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.924985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.925631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.926538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.926538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.927352 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.931972 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:52.944837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7b856-15ea-4113-a81a-10250e7ecf98 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:52.974458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:52.975104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:52.975946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:52.976227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:52.976877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:52.979621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.002777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1241/2493] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/users => generated 281 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:53.014874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.028257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107ecbe-67f6-493d-862a-523558e63583 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.036680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.039120 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.039533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.039825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.041162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.044814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.064187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1242/2494] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:53.065417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.074421 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1253/2495] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:52 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:53.075125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.083657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b858c5-3c43-4582-b82d-6c63eb0f45d7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.198666 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:53.217578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34599c-1d40-4148-adcd-ccf85dc991e3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.230883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.231692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.232211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.232359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] PATH_INFO: `/v3/projects/d56fabeab5724bb8b1d83145af1ea63c/users/4db266d7e1ae49589d607fd51da67640/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.251781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.252443 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.252730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.253502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.254198 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.256452 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.265088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56fabeab5724bb8b1d83145af1ea63c, user_id=4db266d7e1ae49589d607fd51da67640, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.268361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.276519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1243/2496] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:53.276519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.289645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d78215-5cd6-47c6-9105-d60329d56e13 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.295647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1254/2497] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:53 2022] PUT /identity/v3/projects/d56fabeab5724bb8b1d83145af1ea63c/users/4db266d7e1ae49589d607fd51da67640/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:53.296582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.298387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.375811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:53.378349 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68d71-9832-4aac-bdfd-4440fe3ff712 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.384854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.385330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.385494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.385771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.386054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.386169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.386400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.386674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] PATH_INFO: `/v3/projects/b2427414b9594c2a86f59c3614b297ce` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.386798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.387806 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2427414b9594c2a86f59c3614b297ce)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.388616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.390003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.414357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1244/2498] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:53.415127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.428103 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c39a1-f890-4584-b2e0-773cfad62d1d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.435983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.436728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.437116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.437412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.438230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.440058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b47ef-555d-40e3-9aeb-0c0ea037ab99 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.451086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2427414b9594c2a86f59c3614b297ce', 'request_id': 'req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.453749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1245/2499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:53.454700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.454990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.455273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.463922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2427414b9594c2a86f59c3614b297ce', 'request_id': 'req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.471408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2427414b9594c2a86f59c3614b297ce', 'request_id': 'req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.489632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] The token cache is being invalidate because project b2427414b9594c2a86f59c3614b297ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:53.490162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.491371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.494047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1255/2500] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:53 2022] DELETE /identity/v3/projects/b2427414b9594c2a86f59c3614b297ce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:53.495024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.500632 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] MFA Rules not processed for user `4db266d7e1ae49589d607fd51da67640`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:53.502216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.509782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.610919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1246/2501] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:53.612221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.613780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.618358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d627f-bcbe-42fb-bc8f-bdfe095ef8dd admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.623102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.623711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.623999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.624273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.624913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.626548 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385699d-fa89-4669-80e6-a8f20f986da7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.630888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411ff31-c30e-488a-ae62-7c127e9bf8f9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.637625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.637812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1256/2502] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:53.638454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.638454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.638688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] PATH_INFO: `/v3/projects/5e62512d2c234c69b5da85da74fe50c5/users/c39b96b3b2d74a5f9b17e908171b5e23/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.641756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.642346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.642346 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.656507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e62512d2c234c69b5da85da74fe50c5, user_id=c39b96b3b2d74a5f9b17e908171b5e23, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.657119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:53.659254 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.671747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:53.673132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.675949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1247/2503] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:53 2022] PUT /identity/v3/projects/5e62512d2c234c69b5da85da74fe50c5/users/c39b96b3b2d74a5f9b17e908171b5e23/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:53.678607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.680730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.770512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1257/2504] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:53.773845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184742b6-198e-4123-b1ad-fe264aec7f64 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.777947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.780309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.781053 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.781454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.781876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.782615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.784633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7141350-2e56-4602-b744-1f6628468837 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.788962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c64b1b-48fe-4629-b1b1-1bbfaccff600 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.796078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.797090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.797511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.797915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] PATH_INFO: `/v3/projects/090a9576cf6a4341986e32e24592e05c/users/42ce3147f4234c3ebc275bf697e332b6/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.815909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090a9576cf6a4341986e32e24592e05c, user_id=42ce3147f4234c3ebc275bf697e332b6, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.818736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.818975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1248/2505] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:53.820811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84746891-6490-4e15-a217-66863be4d3a8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.821158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84746891-6490-4e15-a217-66863be4d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.821730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84746891-6490-4e15-a217-66863be4d3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.836032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1258/2506] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:53 2022] PUT /identity/v3/projects/090a9576cf6a4341986e32e24592e05c/users/42ce3147f4234c3ebc275bf697e332b6/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:53.836951 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.841284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.855560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-84746891-6490-4e15-a217-66863be4d3a8 None None] MFA Rules not processed for user `c39b96b3b2d74a5f9b17e908171b5e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:53.857232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-84746891-6490-4e15-a217-66863be4d3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.871914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:53.924822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fd7dd-5681-42cc-9908-6cbbd0b7d502 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.930512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.931153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.931605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.931724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] PATH_INFO: `/v3/users/d718e35636464834bc20a6eda0599cfc` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.933037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d718e35636464834bc20a6eda0599cfc)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:53.935017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:53.940261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1249/2507] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:53.942826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.943232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.943747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.969450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:53.971235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:53.980703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1250/2508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:53.981836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:53.992097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660889f-97c7-4eea-956d-54e8d5bee653 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:53.996671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd718e35636464834bc20a6eda0599cfc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.997008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:53.997557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd718e35636464834bc20a6eda0599cfc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.997694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:53.997922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:53.998221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:53.998594 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd718e35636464834bc20a6eda0599cfc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:53.998838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.001052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.011848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd718e35636464834bc20a6eda0599cfc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.028693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1251/2509] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:53 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:54.029885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.031780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.038598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd718e35636464834bc20a6eda0599cfc'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.046272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba332-a74d-499f-abd4-1f6b635289fa admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.048623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:54.052620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.053235 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.054089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.054370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.055175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.057691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.061804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1259/2510] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:53 2022] DELETE /identity/v3/users/d718e35636464834bc20a6eda0599cfc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:54.063457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.065582 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.077079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1252/2511] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:54.078071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.079975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.094736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b173cea-7fed-4b11-b390-8a5a1ddcc6bd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.178960 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.179587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.180489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.180786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.181553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:54.181872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.184292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.190057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a61e5-165d-405b-b42e-c820383daf7d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.208802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.209439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.209754 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.210271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.210842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.214373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1253/2512] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:11:54.216141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.218090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.220419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.244465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f0084-f909-433c-bfa0-ce0449b4a102 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.259783 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.260428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.260715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.260992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.261650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.263354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.265776 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1260/2513] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:54 2022] POST /identity/v3/projects => generated 365 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:54.267359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.284654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1254/2514] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:54.285669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.300694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b1e15-ab2b-4281-9443-9c672a2e5bfd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.382946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc7cd-1e6c-4188-8e67-1e7fe3e73a93 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.388198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.389014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.389398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.389809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] PATH_INFO: `/v3/projects/090a9576cf6a4341986e32e24592e05c/users/42ce3147f4234c3ebc275bf697e332b6/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.391928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090a9576cf6a4341986e32e24592e05c, user_id=42ce3147f4234c3ebc275bf697e332b6, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.394822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.401284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.401943 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.402230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.402553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.403197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.405208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.409802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1261/2515] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:54 2022] PUT /identity/v3/projects/090a9576cf6a4341986e32e24592e05c/users/42ce3147f4234c3ebc275bf697e332b6/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:54.409802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.415691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:54.457305 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1255/2516] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:54 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:54.458257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.502822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaa89-1803-47ac-99fc-ce64234d3223 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.530473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:54.558636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355e4-5c51-4568-a8cb-efa21d91f4b4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.566881 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.568159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.568446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.568724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] PATH_INFO: `/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d/users/1958b9fb8e664b7ebd4f57377a3432b9/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.584408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.585106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.585529 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.585912 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.586553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.586705 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbfc90c1ae64275ae25a074d199bf0d, user_id=1958b9fb8e664b7ebd4f57377a3432b9, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.588786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.589521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.609385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1256/2517] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:54 2022] PUT /identity/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d/users/1958b9fb8e664b7ebd4f57377a3432b9/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:54.612273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.613216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.617859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1262/2518] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:54 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:54.622274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.635551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea1f85-92d6-4e78-bd37-ec165413ca06 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.708261 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.708934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.709451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.710212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.710827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.713074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.730367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046c0fd-5710-46ea-853c-70ee2e95f05b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.738386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.739697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.740192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.740463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] PATH_INFO: `/v3/projects/555943f87fe544d7a45eb6979069230d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.741863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=555943f87fe544d7a45eb6979069230d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.743814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.747765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1263/2519] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:54 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:54.748655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.761485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc585-e171-4ee6-9d23-355909ec9328 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.767630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.768437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.768837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.769183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.769933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.771633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.788731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1264/2520] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:54.790154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.803108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f630-24a2-46ed-933b-270fab8fd6c5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.810162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.811580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.812241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.812721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.813872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.816443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.823843 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555943f87fe544d7a45eb6979069230d', 'request_id': 'req-f02485f7-40b9-4f17-ae4d-685613e83a9d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.833727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555943f87fe544d7a45eb6979069230d', 'request_id': 'req-f02485f7-40b9-4f17-ae4d-685613e83a9d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.835317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1265/2521] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:54.836677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.838726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555943f87fe544d7a45eb6979069230d', 'request_id': 'req-f02485f7-40b9-4f17-ae4d-685613e83a9d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.851896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:54.856104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7efcb-36da-443d-970d-4f599725e096 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.856815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] The token cache is being invalidate because project 555943f87fe544d7a45eb6979069230d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:54.857301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.858341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:54.860358 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1257/2522] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:54 2022] DELETE /identity/v3/projects/555943f87fe544d7a45eb6979069230d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:54.865519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.865519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.938227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.939343 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.939822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.940283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.941292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.944827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.949317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02485f7-40b9-4f17-ae4d-685613e83a9d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.955319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:54.956419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.956784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.957074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] PATH_INFO: `/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d/users/1958b9fb8e664b7ebd4f57377a3432b9/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:54.959276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbfc90c1ae64275ae25a074d199bf0d, user_id=1958b9fb8e664b7ebd4f57377a3432b9, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:54.960482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1266/2523] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:54.961889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:54.964130 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:54.965995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76744139-596d-4af8-b0c8-335cee055eaf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:54.985967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1258/2524] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:54 2022] PUT /identity/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d/users/1958b9fb8e664b7ebd4f57377a3432b9/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:54.987686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e6037-a4a1-47ff-8afc-7fdd539e8c45 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:54.989720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:54.994568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:54.994857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.001663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:55.032375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:55.033651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.083458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.084153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.085469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.085786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] PATH_INFO: `/v3/projects/6e91ebe70756446a80eb778275ff6e2c/users/2f6bb6cba8904d21890fce942441cad8/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.106482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e91ebe70756446a80eb778275ff6e2c, user_id=2f6bb6cba8904d21890fce942441cad8, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.116342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab58f0-5add-4268-841f-9bf7dde26dc5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.116543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1259/2525] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:55.116543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.116543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.134540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572dc8b4-1310-45d5-a17d-4c3b01ceb4e0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.138501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1267/2526] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:54 2022] PUT /identity/v3/projects/6e91ebe70756446a80eb778275ff6e2c/users/2f6bb6cba8904d21890fce942441cad8/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:55.140161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.140656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.140936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.175043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] MFA Rules not processed for user `1958b9fb8e664b7ebd4f57377a3432b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:55.176229 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.230697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.230948 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.231383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.232060 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.233418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.235397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b722dd1-2653-4444-ab68-102350f411e5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.251262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1260/2527] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:55.252762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a96ea-8916-4395-a188-debc954c9872 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.253115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a96ea-8916-4395-a188-debc954c9872 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.253455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a96ea-8916-4395-a188-debc954c9872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.265390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1268/2528] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:55.266217 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.283496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-724a96ea-8916-4395-a188-debc954c9872 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:55.285347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-724a96ea-8916-4395-a188-debc954c9872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.288139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabc52f-c80a-462d-911d-b535e0cf4f6e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.294313 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.294927 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.295233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.295509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.296087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1261/2529] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:55.296209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.297107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a96ea-8916-4395-a188-debc954c9872 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.297891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.310847 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a96ea-8916-4395-a188-debc954c9872 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.317033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1269/2530] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:55.318246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.321076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.322046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.322337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.322617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] PATH_INFO: `/v3/projects/7bae9c2289cf4e6f9916b240ba324c5c/users/6c942cfb5dd74483b5bd749e5f29c471/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.323517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.346709 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bae9c2289cf4e6f9916b240ba324c5c, user_id=6c942cfb5dd74483b5bd749e5f29c471, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.349488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.369292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:55.382983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1262/2531] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:55 2022] PUT /identity/v3/projects/7bae9c2289cf4e6f9916b240ba324c5c/users/6c942cfb5dd74483b5bd749e5f29c471/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:55.384269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.428158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2a322-e156-4fa1-9c11-b9185a999530 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.470864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:55.509073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.510239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.511285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.511922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.529652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5db2a-a568-4a24-b3da-c34225321bf3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.535148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.536101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.536644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.537081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.537950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:11:55.538672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.540507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.555558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1263/2532] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:11:55.557196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.573423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050e75-b55f-4a1b-9d9c-8475fcb27de6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.579669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.580588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.581098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.581569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] PATH_INFO: `/v3/projects/6e91ebe70756446a80eb778275ff6e2c/users/2f6bb6cba8904d21890fce942441cad8/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.584238 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e91ebe70756446a80eb778275ff6e2c, user_id=2f6bb6cba8904d21890fce942441cad8, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.587648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907c457-7edf-43a4-b093-31ee556555bf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.610617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.610617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.611593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1264/2533] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:55 2022] PUT /identity/v3/projects/6e91ebe70756446a80eb778275ff6e2c/users/2f6bb6cba8904d21890fce942441cad8/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:55.613656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6e50-a8a6-41b0-897e-a67acea29f1c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.614163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6e50-a8a6-41b0-897e-a67acea29f1c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.614629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6e50-a8a6-41b0-897e-a67acea29f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.619301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1270/2534] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:55.620534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.633446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bad7c-33f8-4a86-8a1a-e4d8b8ded9c9 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.647836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1fbf6e50-a8a6-41b0-897e-a67acea29f1c None None] MFA Rules not processed for user `6c942cfb5dd74483b5bd749e5f29c471`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:55.649218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1fbf6e50-a8a6-41b0-897e-a67acea29f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.723471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.724543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.725054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.725638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.726580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.728828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.739732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1265/2535] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:55.742678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.743430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.743602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.771795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1271/2536] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/projects => generated 357 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:55.776102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.801542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] MFA Rules not processed for user `2f6bb6cba8904d21890fce942441cad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:55.804428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:55.869262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55bb01-4727-4a0d-808b-cf789907fa6a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:55.875515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:55.876584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:55.877074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:55.877745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:55.878330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:55.880760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:55.898018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:55.926622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1272/2537] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:55.932288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.959983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:55.983611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1266/2538] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:55.985250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:55.999523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338daea4-c9d7-404c-9c08-ae1e6a304971 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.022991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686babca-8279-49dd-b47f-13f9aa4c5d06 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.028374 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.029358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.029973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.030539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.032975 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:56.067774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.068648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.069386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.069674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.070686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.072458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.103476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1267/2539] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:55 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:56.103476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.114701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ab64-0030-4f7b-b8c0-84186cd9997f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.123483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.124370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.124791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.125191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.126911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.127715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.129641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.130322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.156641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1273/2540] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:56.159236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1268/2541] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:56 2022] POST /identity/v3/users => generated 299 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:56.159675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.160743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.164660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:56.181662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4e87e-b0eb-4a1c-a0bb-ba3a6f7bcf1f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.187924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b66e9-d8b9-4b29-8ffe-2711eabf4144 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.189744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.190819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.191256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.191678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.193003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.196233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.197651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.198459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.199734 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.200757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.224298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1269/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:56.226116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.253424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c317c6a-1503-46bd-a634-bf485bfaeff9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.270607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.271885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.274357 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.274924 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.275886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.279565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.306026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1270/2543] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:56.314030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.336397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.338944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.343974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fb648-f49b-40b1-ac45-3f0415458385 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.352947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1274/2544] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:56.354283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.360453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.363805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.364009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.364486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.366323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.370444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.371735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30863898-5408-46cc-90c6-e64d9d71130b service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.388256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.391294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.391752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.392110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] PATH_INFO: `/v3/projects/101c3e96450c492e9a6bd55840a7dcc6/users/d7cdc423632b46558b951d7e7e90afca/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.399538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1271/2545] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:56.406251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.431027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0655b-183c-4472-ba12-ce543adb5254 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.439212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101c3e96450c492e9a6bd55840a7dcc6, user_id=d7cdc423632b46558b951d7e7e90afca, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.445362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.446063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.446353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.446626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] PATH_INFO: `/v3/projects/5521770de7e8440092bff9cca7460eea/users/3eded341c7864cefa2db3a6c3077da4e/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.448172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.487177 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1275/2546] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:56 2022] PUT /identity/v3/projects/101c3e96450c492e9a6bd55840a7dcc6/users/d7cdc423632b46558b951d7e7e90afca/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:56.491124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.491441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5521770de7e8440092bff9cca7460eea, user_id=3eded341c7864cefa2db3a6c3077da4e, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.494340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14424153-2aa1-468f-922e-30cd431c9259 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.522819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1272/2547] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:56 2022] PUT /identity/v3/projects/5521770de7e8440092bff9cca7460eea/users/3eded341c7864cefa2db3a6c3077da4e/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:56.532835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.535001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.536743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.544858 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:56.621546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] MFA Rules not processed for user `3eded341c7864cefa2db3a6c3077da4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:56.624828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:56.671842 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e408ae-5828-4211-a9e6-9bbf5ee27d63 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.848316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:56.856190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1273/2548] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:11:56.857239 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.857458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.858329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.858672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.859124 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.859719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.873056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.882677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61480e47-767f-4105-9ad9-94a3bae343ac None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.889748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.891087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.891657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.892158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.893740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.896502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca73e5-c1ed-485a-802d-4b06d16ee6b1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.902919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1276/2549] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:56.903795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.915667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c34c-99e2-4d8d-86ca-3240d2c5ad1a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:56.931208 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:56.932645 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.933257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.933743 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] PATH_INFO: `/v3/projects/101c3e96450c492e9a6bd55840a7dcc6/users/d7cdc423632b46558b951d7e7e90afca/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:56.936695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=101c3e96450c492e9a6bd55840a7dcc6, user_id=d7cdc423632b46558b951d7e7e90afca, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:56.940053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:56.961939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1277/2550] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:56 2022] PUT /identity/v3/projects/101c3e96450c492e9a6bd55840a7dcc6/users/d7cdc423632b46558b951d7e7e90afca/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:56.964232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1274/2551] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:11:56 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:11:56.965467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:56.973726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:56.974460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:56.975405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.050633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] MFA Rules not processed for user `d7cdc423632b46558b951d7e7e90afca`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:57.052029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.082056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258dc0-4f02-4211-b7ff-25c313e416ec admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.199058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:57.203129 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1275/2552] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:11:57.203926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.214005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.220659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.221147 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.221769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.222531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.228182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.292670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1278/2553] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:11:56 2022] POST /identity/v3/users => generated 285 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:11:57.293584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.333376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715b01-ea83-4c75-a4cd-74ab0b2659a9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.351644 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.352445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.352920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.353518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] PATH_INFO: `/v3/users/c39b96b3b2d74a5f9b17e908171b5e23` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.356971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39b96b3b2d74a5f9b17e908171b5e23)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.359189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.419017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f735d-d723-4e5c-801f-06613c76cd17 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.430356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.431174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.431472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.431785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.454889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.456662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.463291 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39b96b3b2d74a5f9b17e908171b5e23'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.463965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39b96b3b2d74a5f9b17e908171b5e23'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.464603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39b96b3b2d74a5f9b17e908171b5e23'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.467918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1279/2554] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:57.468710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.472689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39b96b3b2d74a5f9b17e908171b5e23'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.500156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:57.510121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427df85-c3e8-4582-a3b3-88a380c50bd6 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.513188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39b96b3b2d74a5f9b17e908171b5e23'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.518522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.519300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.519686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.520095 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.520905 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.523884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4c536-0f41-4a52-8be0-fb3e7147ac18 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.550711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1276/2555] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:57 2022] DELETE /identity/v3/users/c39b96b3b2d74a5f9b17e908171b5e23 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:57.551893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1280/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:57.552516 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.556168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.556534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.556887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.557020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.580770 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:57.583197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.708034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511295af-7b4e-4f9e-837f-9ccd887de587 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.721212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.723583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.724273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.724558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] PATH_INFO: `/v3/projects/66263caab08d4a19900b4276432b003b/users/5d8c962b66e34df0b693467c5b4220d8/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.740450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1281/2557] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:57.740450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.740450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.754928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5b68a-53c7-4770-876d-11cf5d1ef785 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.758841 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66263caab08d4a19900b4276432b003b, user_id=5d8c962b66e34df0b693467c5b4220d8, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.761852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.762039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.762884 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.763243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.763596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] PATH_INFO: `/v3/users/6c942cfb5dd74483b5bd749e5f29c471` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.764832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c942cfb5dd74483b5bd749e5f29c471)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:57.766969 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:57.783935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1277/2558] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:57 2022] PUT /identity/v3/projects/66263caab08d4a19900b4276432b003b/users/5d8c962b66e34df0b693467c5b4220d8/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:11:57.783935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.787383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.874018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc33d5-aff4-4bdc-ad5f-00ccc61ab3d4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:57.877893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:57.878495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c942cfb5dd74483b5bd749e5f29c471'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.878985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:57.878985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:57.879172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:57.882839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c942cfb5dd74483b5bd749e5f29c471'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.884824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c942cfb5dd74483b5bd749e5f29c471'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.899282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c942cfb5dd74483b5bd749e5f29c471'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.946741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:57.953683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c942cfb5dd74483b5bd749e5f29c471'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:57.995199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1282/2559] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:57 2022] DELETE /identity/v3/users/6c942cfb5dd74483b5bd749e5f29c471 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:57.996837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:57.999246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:57.999494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:58.001979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af0bc4-4618-4f8c-863d-59f8226a0748 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:58.026368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1278/2560] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:58.036075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877834cd-f182-4458-8bdc-d790be377200 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.036372 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877834cd-f182-4458-8bdc-d790be377200 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.036650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877834cd-f182-4458-8bdc-d790be377200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.113769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-877834cd-f182-4458-8bdc-d790be377200 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:58.115298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-877834cd-f182-4458-8bdc-d790be377200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:58.157197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:58.196783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa44b-8679-4270-8b7c-e58114630943 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:58.208122 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:58.210058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.210473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.211711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.212558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:58.216741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:58.248322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1283/2561] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:11:58.250774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:58.275727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21d8d-d07f-49cf-982d-1d0eac17df76 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:58.293262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:58.294145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.294527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.295665 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] PATH_INFO: `/v3/projects/66263caab08d4a19900b4276432b003b/users/5d8c962b66e34df0b693467c5b4220d8/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.299877 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] RBAC: Authorizing `identity:create_grant(project_id=66263caab08d4a19900b4276432b003b, user_id=5d8c962b66e34df0b693467c5b4220d8, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:58.308520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989ddb6-0d23-460e-9e1e-c661482d821b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:58.340531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1284/2562] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:11:58 2022] PUT /identity/v3/projects/66263caab08d4a19900b4276432b003b/users/5d8c962b66e34df0b693467c5b4220d8/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:58.342313 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1279/2563] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:58.351715 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.353240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.353625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.414840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877834cd-f182-4458-8bdc-d790be377200 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:58.417350 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-877834cd-f182-4458-8bdc-d790be377200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:58.430276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] MFA Rules not processed for user `5d8c962b66e34df0b693467c5b4220d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:11:58.432437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:58.612755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1285/2564] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:11:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:11:58.621953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877834cd-f182-4458-8bdc-d790be377200 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:58.633541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:58.645018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:58.744250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:58.745037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.745182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.745433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.786829 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cddcad-26e2-4ae1-9eba-72921eeb04ea None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:58.798102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:58.799309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.799497 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.800026 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.814286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:58.815982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:58.821886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:58.846174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1286/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:58.872348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:58.873485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:58.893749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1280/2566] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:58.970916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:58.981930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ce45-72b4-444f-940a-8bbdecd7a710 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:58.987688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:58.988680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:58.989178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:58.989945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:58.992211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:59.078747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:59.080395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:59.095846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1287/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:11:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:11:59.436753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:59.438359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:59.454504 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf97d-c310-45f5-9d2c-fce48c5e330c service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:59.460569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:59.461245 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:59.461568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:59.461826 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] PATH_INFO: `/v3/projects/5e62512d2c234c69b5da85da74fe50c5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:59.462965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e62512d2c234c69b5da85da74fe50c5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:59.465263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:59.489953 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:11:59.530616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e62512d2c234c69b5da85da74fe50c5', 'request_id': 'req-6974a2c6-29a0-4488-8533-0bee2a66d81e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.545251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e62512d2c234c69b5da85da74fe50c5', 'request_id': 'req-6974a2c6-29a0-4488-8533-0bee2a66d81e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.557908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e62512d2c234c69b5da85da74fe50c5', 'request_id': 'req-6974a2c6-29a0-4488-8533-0bee2a66d81e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.593556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] The token cache is being invalidate because project 5e62512d2c234c69b5da85da74fe50c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:11:59.593905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.594585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.596177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1281/2568] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:59 2022] DELETE /identity/v3/projects/5e62512d2c234c69b5da85da74fe50c5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:59.601402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:59.622115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52568167-e153-4441-8a2f-e10f98cf55f5 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:59.718301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:59.719025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:59.719309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:59.719585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] PATH_INFO: `/v3/users/2f6bb6cba8904d21890fce942441cad8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:59.721643 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6bb6cba8904d21890fce942441cad8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:59.723993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:59.778609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:11:59.789190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:11:59.807018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974a2c6-29a0-4488-8533-0bee2a66d81e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:11:59.823310 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:11:59.823982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:11:59.824269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:11:59.824554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] PATH_INFO: `/v3/projects/7bae9c2289cf4e6f9916b240ba324c5c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:11:59.827874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bae9c2289cf4e6f9916b240ba324c5c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:11:59.832629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:11:59.848234 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6bb6cba8904d21890fce942441cad8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.850047 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6bb6cba8904d21890fce942441cad8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.852679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6bb6cba8904d21890fce942441cad8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.872928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6bb6cba8904d21890fce942441cad8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.908525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6bb6cba8904d21890fce942441cad8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.939867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1288/2569] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:11:59 2022] DELETE /identity/v3/users/2f6bb6cba8904d21890fce942441cad8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:11:59.972220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bae9c2289cf4e6f9916b240ba324c5c', 'request_id': 'req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:11:59.973638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bae9c2289cf4e6f9916b240ba324c5c', 'request_id': 'req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.001156 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bae9c2289cf4e6f9916b240ba324c5c', 'request_id': 'req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.034842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] The token cache is being invalidate because project 7bae9c2289cf4e6f9916b240ba324c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:00.035148 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.035941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.039340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1282/2570] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:11:59 2022] DELETE /identity/v3/projects/7bae9c2289cf4e6f9916b240ba324c5c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:00.045430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:00.047823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:00.119126 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:00.296426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8087fd1-25ca-49f4-97e8-82cfaea14ed8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:00.323091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:00.323937 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:00.324366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:00.329589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] PATH_INFO: `/v3/users/d7cdc423632b46558b951d7e7e90afca` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:00.331104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7cdc423632b46558b951d7e7e90afca)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:00.333255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:00.421807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:00.423439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:00.496358 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7cdc423632b46558b951d7e7e90afca'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.498824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7cdc423632b46558b951d7e7e90afca'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.500481 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7cdc423632b46558b951d7e7e90afca'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.520660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7cdc423632b46558b951d7e7e90afca'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.580013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6bbf82b2-6671-40f5-b20f-4324b25ba0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7cdc423632b46558b951d7e7e90afca'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.615070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1289/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:00 2022] DELETE /identity/v3/users/d7cdc423632b46558b951d7e7e90afca => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:00.631765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5061a-2ca5-45d4-810b-ac2aad5403c2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:00.715507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:00.800932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:00.802312 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:00.802997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:00.803919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] PATH_INFO: `/v3/projects/6e91ebe70756446a80eb778275ff6e2c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:00.809670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e91ebe70756446a80eb778275ff6e2c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:00.813434 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:00.945490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e91ebe70756446a80eb778275ff6e2c', 'request_id': 'req-39c79e59-e267-4c97-aa53-315691b3063e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.963072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e91ebe70756446a80eb778275ff6e2c', 'request_id': 'req-39c79e59-e267-4c97-aa53-315691b3063e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:00.974667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e91ebe70756446a80eb778275ff6e2c', 'request_id': 'req-39c79e59-e267-4c97-aa53-315691b3063e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.017758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] The token cache is being invalidate because project 6e91ebe70756446a80eb778275ff6e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:01.017758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.020033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-39c79e59-e267-4c97-aa53-315691b3063e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.028772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1283/2572] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:00 2022] DELETE /identity/v3/projects/6e91ebe70756446a80eb778275ff6e2c => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:01.029778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.030167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.030525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.073837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:01.075775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.175071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1290/2573] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:01.186178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.186469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.186932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.216818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:01.218116 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.239743 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1284/2574] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:01.248898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.266617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a9715-f8d5-49cb-a8cc-51a1f64cc2c5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.273222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.273911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.274174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.274639 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.279617 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:01.281082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.283263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.292708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.296965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.305491 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef2ab-6321-481a-8a36-ee24b9911330 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.320618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.321284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.321637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.321971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] PATH_INFO: `/v3/projects/101c3e96450c492e9a6bd55840a7dcc6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.329445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101c3e96450c492e9a6bd55840a7dcc6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.331514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.331584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1291/2575] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:01.332429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.354491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f0a-c1e9-4916-85f2-825525d0d54a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.365210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.366446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.366732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.367009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.368707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.369619 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.414659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:01.428459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1292/2576] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:01.430002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.450236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101c3e96450c492e9a6bd55840a7dcc6', 'request_id': 'req-f5d0ee05-199b-4ad0-805d-ce53ea150a89'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.462698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101c3e96450c492e9a6bd55840a7dcc6', 'request_id': 'req-f5d0ee05-199b-4ad0-805d-ce53ea150a89'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.472894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101c3e96450c492e9a6bd55840a7dcc6', 'request_id': 'req-f5d0ee05-199b-4ad0-805d-ce53ea150a89'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.498797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] The token cache is being invalidate because project 101c3e96450c492e9a6bd55840a7dcc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:01.500337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.504930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5d0ee05-199b-4ad0-805d-ce53ea150a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:01.513083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1285/2577] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:01 2022] DELETE /identity/v3/projects/101c3e96450c492e9a6bd55840a7dcc6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:01.514761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1399e-88fe-408d-811e-4bc8b650c5ea None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.515132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1399e-88fe-408d-811e-4bc8b650c5ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.515489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1399e-88fe-408d-811e-4bc8b650c5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.546685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d5a1399e-88fe-408d-811e-4bc8b650c5ea None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:01.548292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d5a1399e-88fe-408d-811e-4bc8b650c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.555134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19900b22-04ee-49d7-b0bb-3d4c1e96e21f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.558124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1286/2578] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:01.567508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.568556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.569074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.569491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.570724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.573012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.577217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.577512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.577894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.604529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:01.609339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1293/2579] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:01.610893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.613157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.615035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:01.624690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1287/2580] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:01.629595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.636514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67a36e-2a64-4f7d-ae54-e2b7ed05f3df admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.643769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c429e5-7e71-458b-9f36-32b000ebd3ea None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.649023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.649993 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.650279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.650558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.650955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.651271 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.651584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.651873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.652153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.652833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:01.653132 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.653398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.655094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.675766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1294/2581] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:01.686638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.686865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1288/2582] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:12:01.687991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.700194 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36169e3-c8ed-4ff9-ac9f-e7b8037182f3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.704754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29d7cd-b93b-4da9-a590-8fa5acafcc7a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.710642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.712863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.713155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.713833 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.714828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.715225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.715327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.715675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.715856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.716509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.717833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.718469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.737914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1295/2583] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:01.740000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:01.743018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.766489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1289/2584] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:01.767338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.769308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19734860-11b4-4456-9960-8e692dc001e1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.865570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9520b81-9716-4512-b334-42a770f3bf13 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:01.871091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.871848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.872153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.873534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.875176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.878244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193667e4-94b0-42dd-be0f-0a8b2aae40f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.884845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:01.885530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.885843 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.886127 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] PATH_INFO: `/v3/projects/bccfa02fefef420ca2f594498858e2f5/users/815b9f3a1b3740e4b64d2973d85f92e8/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.922397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccfa02fefef420ca2f594498858e2f5, user_id=815b9f3a1b3740e4b64d2973d85f92e8, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:01.927314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1290/2585] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:01.927314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:01.927314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:01.927314 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:01.932013 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:01.958007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1296/2586] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:01 2022] PUT /identity/v3/projects/bccfa02fefef420ca2f594498858e2f5/users/815b9f3a1b3740e4b64d2973d85f92e8/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:01.959283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:01.974359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] MFA Rules not processed for user `42ce3147f4234c3ebc275bf697e332b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:01.975633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.035007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:02.052210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c17253-315a-46b2-8f89-92f1f916a893 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.063431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.064191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.064562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.064901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.065631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.067430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ba08b-b66d-456d-8227-8c76e1df0963 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.080003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1297/2587] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:02.081567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.081970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.082257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.101031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1291/2588] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:02.103379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.120915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f85d6e-536e-4f3a-b7ed-e5b7edf6d25d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.123320 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] MFA Rules not processed for user `815b9f3a1b3740e4b64d2973d85f92e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:02.124682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.133834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.134477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.134767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.135046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.135708 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.137788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.158464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1292/2589] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:02.159716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.161669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.211740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1298/2590] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:02.213039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.226605 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c24b2-5846-4f86-9a45-5e0c62e6bf10 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.232315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.233255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.233427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.233845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] PATH_INFO: `/v3/projects/56772c6e93e9411f90c9b6dca26779cc/users/7d628776253a457aaea4d5d56430723f/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.234927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:02.261853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e49e8-f0fb-4890-914a-e72286fd4192 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.268813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56772c6e93e9411f90c9b6dca26779cc, user_id=7d628776253a457aaea4d5d56430723f, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.273956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.275977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.277243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.278126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.278417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.281576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.305171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1299/2591] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:02 2022] PUT /identity/v3/projects/56772c6e93e9411f90c9b6dca26779cc/users/7d628776253a457aaea4d5d56430723f/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:02.306584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.385940 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.388756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493005cf-aa33-43f4-9618-53a172b734dd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.407079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1293/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:02.408217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb493-7186-458b-82c6-690376982c25 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.408508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb493-7186-458b-82c6-690376982c25 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.408887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb493-7186-458b-82c6-690376982c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.410627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f5237-b58f-4897-9933-498733fd6c08 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.418078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.418078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.418277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.418538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.419161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.421370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.447886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-039cb493-7186-458b-82c6-690376982c25 None None] MFA Rules not processed for user `7d628776253a457aaea4d5d56430723f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:02.449293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-039cb493-7186-458b-82c6-690376982c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.457057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1300/2593] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:02.461852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.528925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:02.563620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29d63d-fa81-4c24-ab74-3ad3ffd4fe97 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.563802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1294/2594] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:02.570523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.571839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.572279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb493-7186-458b-82c6-690376982c25 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.572744 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.573141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.573999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.574716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-039cb493-7186-458b-82c6-690376982c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.577184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.619002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1301/2595] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:02.620826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.639045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca77c8-7255-44bf-b53a-bcdbe4c157a8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.647861 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.649154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.649552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.649956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.650723 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.653236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.668476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1302/2596] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:02.668660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb493-7186-458b-82c6-690376982c25 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.670480 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.715167 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ff71b-2dfe-4644-a6c1-4fa3cdfb9553 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.721228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:02.727584 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.728224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.728508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.728786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.729443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.731134 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.741698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.742494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.742786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.743355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.751256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1303/2597] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:02.753025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.755103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:02.768724 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42273-4f48-453b-b954-f8dc606f7e31 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.774180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.775153 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.775586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.776005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] PATH_INFO: `/v3/users/4db266d7e1ae49589d607fd51da67640` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.777317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db266d7e1ae49589d607fd51da67640)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.779468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.822651 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.824155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.836642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1295/2598] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:02.837495 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:02.846090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db266d7e1ae49589d607fd51da67640'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:02.848160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db266d7e1ae49589d607fd51da67640'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:02.848796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db266d7e1ae49589d607fd51da67640'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:02.849125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33a90b-dc1f-4e5a-a677-c5b561514ade service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:02.856188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:02.858416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.858416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.858416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] PATH_INFO: `/v3/projects/ad75aa7692384a258d015e0eaf662123/users/dfa763f4cf5e4909bad734a67552b199/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.879547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db266d7e1ae49589d607fd51da67640'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:02.884849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad75aa7692384a258d015e0eaf662123, user_id=dfa763f4cf5e4909bad734a67552b199, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:02.887460 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777f105-7d47-4d4a-96c2-0a4e265c19f4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:02.900660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-849ab558-4f81-43b3-b1f2-df16828ac210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db266d7e1ae49589d607fd51da67640'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:02.917123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1296/2599] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:02 2022] PUT /identity/v3/projects/ad75aa7692384a258d015e0eaf662123/users/dfa763f4cf5e4909bad734a67552b199/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:02.927479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1304/2600] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:02 2022] DELETE /identity/v3/users/4db266d7e1ae49589d607fd51da67640 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:02.931020 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.931158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.931487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:02.932560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:02.932857 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:02.933136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.005234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] MFA Rules not processed for user `dfa763f4cf5e4909bad734a67552b199`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:03.007437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:03.140781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1297/2601] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:03.145574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.148911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:03.154593 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:03.158546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:03.244384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1305/2602] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:03.245878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eeba5-1214-4357-8cf4-e6958c50b1aa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.251464 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.252608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.252938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.253211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.254038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.254936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.257027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.258204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:03.263771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:03.301784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1298/2603] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:03 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:03.304997 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:03.306697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.370073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f506d-82a1-4cdd-a3ac-d74af45b3f5e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.483662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189960ee-8bc7-41bd-9ca7-abb0b1035c1a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.493401 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.494315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.496318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.496318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.498625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.500430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.501967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.502378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.502735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.503088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.523454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.526525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.551742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1306/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:03.555693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1299/2605] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:03 2022] POST /identity/v3/users => generated 296 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:03.561240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.581260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1674e-321c-43f0-a78e-27cd4ae404a5 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.588646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.589836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.590662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.591203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.592010 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.594422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.621207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1307/2606] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:03.621787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.638796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0bb7-74d3-46d6-a8eb-e8c96c7c100d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.646571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.648157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.648914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.649300 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] PATH_INFO: `/v3/projects/aa76843d608f4117a1de39ad15ebccd2/users/0f1585eb50b24adab2ef6febdf06da8f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.679602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa76843d608f4117a1de39ad15ebccd2, user_id=0f1585eb50b24adab2ef6febdf06da8f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.683566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.704033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1300/2607] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:03 2022] PUT /identity/v3/projects/aa76843d608f4117a1de39ad15ebccd2/users/0f1585eb50b24adab2ef6febdf06da8f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:03.707958 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.722819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48019e-e85a-4ccd-9180-7a3cdfc0d8f2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.816669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.820385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.822784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.823591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.824681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.827804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd62f78-fc47-4570-9c7a-7cd4b519848e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.844284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1308/2608] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:03.845877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:03.861485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395ebf-b616-4430-9b07-f6ff737dadb9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:03.870799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:03.872529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.873051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.873998 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] PATH_INFO: `/v3/projects/aa76843d608f4117a1de39ad15ebccd2/users/0f1585eb50b24adab2ef6febdf06da8f/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.876445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa76843d608f4117a1de39ad15ebccd2, user_id=0f1585eb50b24adab2ef6febdf06da8f, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:03.879974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:03.901820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1301/2609] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:03 2022] PUT /identity/v3/projects/aa76843d608f4117a1de39ad15ebccd2/users/0f1585eb50b24adab2ef6febdf06da8f/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:03.908886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:03.909326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:03.909798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:03.972014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] MFA Rules not processed for user `0f1585eb50b24adab2ef6febdf06da8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:03.973655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.058531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:04.061173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.075728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:04.136489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1309/2610] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:04.150244 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:04.153348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.182255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c7df2-6429-4240-8a93-28070590397e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:04.190493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:04.192043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:04.192043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:04.192043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] PATH_INFO: `/v3/projects/d56fabeab5724bb8b1d83145af1ea63c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:04.193049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56fabeab5724bb8b1d83145af1ea63c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:04.195832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:04.266806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:04.277896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56fabeab5724bb8b1d83145af1ea63c', 'request_id': 'req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.286231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56fabeab5724bb8b1d83145af1ea63c', 'request_id': 'req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.287319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1939c-a9aa-4071-a254-896a7920a627 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:04.293763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56fabeab5724bb8b1d83145af1ea63c', 'request_id': 'req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.312215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] The token cache is being invalidate because project d56fabeab5724bb8b1d83145af1ea63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:04.312729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.314068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fa2cac5d-19a3-44f6-84b3-fd2d78ef24f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.316557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1302/2611] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:04 2022] DELETE /identity/v3/projects/d56fabeab5724bb8b1d83145af1ea63c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:04.318100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:04.318701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:04.319966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:04.356077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] MFA Rules not processed for user `deb44b514dbf425aa108690d647179f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:04.357777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.371284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:04.371916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:04.372204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:04.372486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:04.374373 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.435698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1303/2612] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:04.439906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:04.439906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.511429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:04.512959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:04.527048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1310/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:04.540176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf59f-e6a0-48e9-86e4-dd98c8fc9a92 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:04.546105 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:04.546761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:04.547044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:04.547411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] PATH_INFO: `/v3/users/1958b9fb8e664b7ebd4f57377a3432b9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:04.548999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1958b9fb8e664b7ebd4f57377a3432b9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:04.551343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:04.623784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1958b9fb8e664b7ebd4f57377a3432b9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.624628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1958b9fb8e664b7ebd4f57377a3432b9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.625537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1958b9fb8e664b7ebd4f57377a3432b9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.634959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1958b9fb8e664b7ebd4f57377a3432b9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.639397 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:04.642799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.651617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1958b9fb8e664b7ebd4f57377a3432b9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:04.653568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:04.656309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345f86-cc3c-4cad-adfa-15c5187ebad2 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:04.661335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:04.662168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:04.662411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:04.663487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:04.667104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:04.685204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1304/2614] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:04 2022] DELETE /identity/v3/users/1958b9fb8e664b7ebd4f57377a3432b9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:04.752762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:04.767673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:04.770409 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87874ea6-ea74-40c2-801b-256df180e587 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:04.791450 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1311/2615] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:05.050897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.052451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:05.168149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a8d2a-7366-48b4-8f53-de7490d6b104 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.178236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.179827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.180195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.181024 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] PATH_INFO: `/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.182697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbfc90c1ae64275ae25a074d199bf0d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.186118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.274456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbfc90c1ae64275ae25a074d199bf0d', 'request_id': 'req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:05.284122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbfc90c1ae64275ae25a074d199bf0d', 'request_id': 'req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:05.307115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbfc90c1ae64275ae25a074d199bf0d', 'request_id': 'req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:05.338829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] The token cache is being invalidate because project 6bbfc90c1ae64275ae25a074d199bf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:05.339420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:05.345723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c38edbb-8ca5-4006-8ed4-ba8d1a205cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:05.348035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1305/2616] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:05 2022] DELETE /identity/v3/projects/6bbfc90c1ae64275ae25a074d199bf0d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:05.352922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.353389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.353458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.405471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:05.407929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:05.571813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1312/2617] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:05.573591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.573921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.574204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.613752 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:05.616370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:05.627507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1306/2618] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:05.631042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.650721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800f767-10a3-4f60-acb7-d7657feb8bd0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.660093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.660698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.660983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.661490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.662112 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:05.662469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.667698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.682259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1313/2619] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:05.688118 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.712013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26955acd-f25f-4745-8ec5-a7206ef69398 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.725789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.726678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.727187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.727392 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.728075 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.730134 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.769374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1307/2620] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:05 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:05.774561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.786873 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3a35-8838-4d9c-8e43-f231d7593c63 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.850615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.851262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.851566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.851828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.852461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.854437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.880286 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1314/2621] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:05 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:05.885223 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.900122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ef51-9dce-4fae-a5fd-a125ff3659dd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.908502 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.909241 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.909529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.909837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.910865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.912693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.925758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1308/2622] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:05.928850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.946403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4241a1-4409-4527-807e-b37f72debea9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.953578 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.954237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:05.954518 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:05.954800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:05.955446 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:05.957297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73bb3-90e1-4852-8b94-3c8c21110349 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:05.969753 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1315/2623] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:05.974046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:05.988929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd7461-921f-4d7b-a338-099b1ab2d462 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:05.998682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:05.999682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.000220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.000667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] PATH_INFO: `/v3/projects/ef35c7f3b53f45f1a6478ccfbae55020/users/56a65d3ba6df437eae208e9491d557cf/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.020036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef35c7f3b53f45f1a6478ccfbae55020, user_id=56a65d3ba6df437eae208e9491d557cf, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:06.022848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:06.041789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1309/2624] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:05 2022] PUT /identity/v3/projects/ef35c7f3b53f45f1a6478ccfbae55020/users/56a65d3ba6df437eae208e9491d557cf/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:06.048387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.048713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.048990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.084460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] MFA Rules not processed for user `56a65d3ba6df437eae208e9491d557cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:06.085714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:06.141183 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:06.171952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1316/2625] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:06.188439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:06.190530 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:06.215233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:06.264850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc18c6-46c6-41b6-b852-854260bc6d3e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:06.268629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:06.269258 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.269547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.269955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.280090 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:06.282939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53807429-f597-4d32-bf5c-df56bedf72ce service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:06.293096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1310/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:06.314212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:06.390944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80951d7-24dd-4446-ba71-428fabc4f051 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:06.395501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:06.396133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.396412 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.396685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] PATH_INFO: `/v3/users/56a65d3ba6df437eae208e9491d557cf` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.398420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a65d3ba6df437eae208e9491d557cf)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:06.400321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:06.452532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a65d3ba6df437eae208e9491d557cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:06.453212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a65d3ba6df437eae208e9491d557cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:06.453872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a65d3ba6df437eae208e9491d557cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:06.461967 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a65d3ba6df437eae208e9491d557cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:06.475872 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a65d3ba6df437eae208e9491d557cf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:06.493769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1317/2627] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:06 2022] DELETE /identity/v3/users/56a65d3ba6df437eae208e9491d557cf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:06.495526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.495813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.496142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.516484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:06.517734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:06.713603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1311/2628] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:06.723920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:06.726001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:06.817420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f4225-6817-4fdd-9e93-b3d4ff4dc678 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:06.823677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:06.824155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:06.824439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:06.824947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:06.839781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:06.841597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363e776-34f9-47e1-b014-ad0ab4f981ab service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:06.851998 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1318/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:07.244378 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.246208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:07.286854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03351b9-a9f4-4d69-8cfa-83790e1d85be None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.293041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.293988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.294316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.294761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] PATH_INFO: `/v3/projects/ef35c7f3b53f45f1a6478ccfbae55020` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.299140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef35c7f3b53f45f1a6478ccfbae55020)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.302285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.372780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef35c7f3b53f45f1a6478ccfbae55020', 'request_id': 'req-2addfbee-a9d5-4b58-ac12-0d25b29781f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:07.378015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef35c7f3b53f45f1a6478ccfbae55020', 'request_id': 'req-2addfbee-a9d5-4b58-ac12-0d25b29781f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:07.384625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef35c7f3b53f45f1a6478ccfbae55020', 'request_id': 'req-2addfbee-a9d5-4b58-ac12-0d25b29781f1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:07.415002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] The token cache is being invalidate because project ef35c7f3b53f45f1a6478ccfbae55020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:07.415306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:07.416041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2addfbee-a9d5-4b58-ac12-0d25b29781f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:07.417534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1312/2630] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:07 2022] DELETE /identity/v3/projects/ef35c7f3b53f45f1a6478ccfbae55020 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:07.422825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.423706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.424246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.445980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:07.447191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:07.451433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:07.525507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1319/2631] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:07.535631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.535631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.535631 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.558557 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:07.560106 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:07.569207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1313/2632] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:07.577624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.588622 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d459955-593f-4e21-8342-876cb8a338b6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.594212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.594810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.595091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.595364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.596008 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:07.596307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.597925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.609880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1320/2633] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:07.614104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.624499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e216-6399-414e-bf59-9cb5efc86b0a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.629858 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.630694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.630980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.631259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.631895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.633836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.662079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1314/2634] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:07 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:07.667175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.677142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b7c24-049c-4aac-9eef-f21e917011f7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.736557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.737157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.737440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.737735 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.738370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.741553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.764007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1321/2635] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:07 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:07.768270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.783734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce57a2b-3ff0-4e0a-8acd-e8a095cea7b8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.789470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.790100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.790385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.790657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.791301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.792894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.803959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1315/2636] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:07.808439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.818627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba7527-035c-41db-9d6f-d5e6efc29736 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.823347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.823945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.824227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.824498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.825132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.827021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4903cc-8b07-43f8-b721-73e8f948cf20 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.837377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1322/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:07.841458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:07.847649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:07.852002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fc8ce-9c45-42aa-a379-cca1e87cba40 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:07.858284 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:07.859073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.859563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.859859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] PATH_INFO: `/v3/projects/69ab105950454b6090548a1e2c2389d2/users/a1ceff4beeb44657a19ab5df5f1b8139/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.882272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ab105950454b6090548a1e2c2389d2, user_id=a1ceff4beeb44657a19ab5df5f1b8139, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:07.884983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:07.899250 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1316/2638] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:07 2022] PUT /identity/v3/projects/69ab105950454b6090548a1e2c2389d2/users/a1ceff4beeb44657a19ab5df5f1b8139/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:07.910106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:07.910395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:07.910669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:07.944874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] MFA Rules not processed for user `a1ceff4beeb44657a19ab5df5f1b8139`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:07.946140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.026560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1323/2639] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:08.034006 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:08.037318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.102659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4a3e-8960-4cef-b60a-99ef0fc9ac04 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:08.106299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:08.106895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.107176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.107484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.116002 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:08.117613 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03e2b1-271d-4ff7-ae43-8808c2faebb7 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:08.127739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1317/2640] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:08.151274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:08.157433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:08.225952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893bb2c-3c88-4c81-a6c7-db585c78ba25 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:08.230824 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:08.231579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.231867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.232168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] PATH_INFO: `/v3/users/a1ceff4beeb44657a19ab5df5f1b8139` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.233310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ceff4beeb44657a19ab5df5f1b8139)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:08.235302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:08.287957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ceff4beeb44657a19ab5df5f1b8139'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.289960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ceff4beeb44657a19ab5df5f1b8139'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.290310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ceff4beeb44657a19ab5df5f1b8139'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.304893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ceff4beeb44657a19ab5df5f1b8139'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.318110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ceff4beeb44657a19ab5df5f1b8139'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.337798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1324/2641] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:08 2022] DELETE /identity/v3/users/a1ceff4beeb44657a19ab5df5f1b8139 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:08.340332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.340623 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.340901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.367191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:08.368286 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.438681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:08.440281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.448256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1318/2642] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:08.457994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dc725-63a9-4d66-b42e-ff49c864066d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:08.458650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:08.461736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.470296 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:08.471210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.471589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.472024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] PATH_INFO: `/v3/users/815b9f3a1b3740e4b64d2973d85f92e8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.473559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815b9f3a1b3740e4b64d2973d85f92e8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:08.476359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:08.547384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815b9f3a1b3740e4b64d2973d85f92e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.548371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815b9f3a1b3740e4b64d2973d85f92e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.549876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815b9f3a1b3740e4b64d2973d85f92e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.559870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3bd0-ca5f-4b19-bbb8-5fb138a9049b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:08.560417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815b9f3a1b3740e4b64d2973d85f92e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.563545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:08.564289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.564610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.564854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.566994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.586250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815b9f3a1b3740e4b64d2973d85f92e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:08.589248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:08.590718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:08.591026 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e30c-9c05-43b7-93e7-1527d8c7dc95 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:08.605848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1319/2643] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:08.628537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1325/2644] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:08 2022] DELETE /identity/v3/users/815b9f3a1b3740e4b64d2973d85f92e8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:08.633896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.634194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.634574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.658845 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:08.660080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.691780 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:08.761853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1320/2645] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:08.771819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:08.773560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:08.891611 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7a6fc-0a11-40be-b235-8dd342aa6d43 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:08.895704 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:08.896357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:08.897554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:08.897949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:08.915179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:08.926074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:08.938074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1326/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:09.119001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:09.121637 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:09.143183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662fd96-6021-4f83-a00e-55909a92acde None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:09.151049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:09.151905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:09.152191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:09.152560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] PATH_INFO: `/v3/projects/69ab105950454b6090548a1e2c2389d2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:09.154064 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ab105950454b6090548a1e2c2389d2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:09.169084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:09.239459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ab105950454b6090548a1e2c2389d2', 'request_id': 'req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.249240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ab105950454b6090548a1e2c2389d2', 'request_id': 'req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.260698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ab105950454b6090548a1e2c2389d2', 'request_id': 'req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.274814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] The token cache is being invalidate because project 69ab105950454b6090548a1e2c2389d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:09.275211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.275961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.278209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1321/2647] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:09 2022] DELETE /identity/v3/projects/69ab105950454b6090548a1e2c2389d2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:09.400022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:09.401620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:09.429179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c641f5-3b5c-46f4-b0ee-bf558c136f90 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:09.519231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:09.519897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:09.520180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:09.520641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] PATH_INFO: `/v3/projects/bccfa02fefef420ca2f594498858e2f5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:09.522441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccfa02fefef420ca2f594498858e2f5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:09.524453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:09.580295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccfa02fefef420ca2f594498858e2f5', 'request_id': 'req-ac03d824-057e-44e5-87dc-653af576696d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.588733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccfa02fefef420ca2f594498858e2f5', 'request_id': 'req-ac03d824-057e-44e5-87dc-653af576696d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.596265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccfa02fefef420ca2f594498858e2f5', 'request_id': 'req-ac03d824-057e-44e5-87dc-653af576696d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.614243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] The token cache is being invalidate because project bccfa02fefef420ca2f594498858e2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:09.614540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.615193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.616702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1327/2648] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:09 2022] DELETE /identity/v3/projects/bccfa02fefef420ca2f594498858e2f5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:09.620813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:09.622325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:09.635041 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c5a3-20bf-4e62-afc5-a2ab6b5895b0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:09.703687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:09.704517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:09.704863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:09.705206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] PATH_INFO: `/v3/users/dfa763f4cf5e4909bad734a67552b199` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:09.706410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa763f4cf5e4909bad734a67552b199)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:09.708376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:09.779882 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa763f4cf5e4909bad734a67552b199'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.780547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa763f4cf5e4909bad734a67552b199'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.781165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa763f4cf5e4909bad734a67552b199'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.785828 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:09.792463 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa763f4cf5e4909bad734a67552b199'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.806309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa763f4cf5e4909bad734a67552b199'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:09.824047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1322/2649] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:09 2022] DELETE /identity/v3/users/dfa763f4cf5e4909bad734a67552b199 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:10.117894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:10.119849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:10.137868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03d824-057e-44e5-87dc-653af576696d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:10.187558 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:10.236128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:10.236787 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:10.237140 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:10.237500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] PATH_INFO: `/v3/projects/ad75aa7692384a258d015e0eaf662123` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:10.238688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad75aa7692384a258d015e0eaf662123)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:10.240943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:10.307027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad75aa7692384a258d015e0eaf662123', 'request_id': 'req-8394b459-da7f-4179-9a25-ed87089f6dcf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.316763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad75aa7692384a258d015e0eaf662123', 'request_id': 'req-8394b459-da7f-4179-9a25-ed87089f6dcf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.324902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad75aa7692384a258d015e0eaf662123', 'request_id': 'req-8394b459-da7f-4179-9a25-ed87089f6dcf'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.342503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] The token cache is being invalidate because project ad75aa7692384a258d015e0eaf662123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:10.342799 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.343483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.348306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1328/2650] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:10 2022] DELETE /identity/v3/projects/ad75aa7692384a258d015e0eaf662123 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:10.349271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:10.359620 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:10.376290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad7e0b-ad2e-44ea-a34c-2182854d595a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:10.448563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:10.449335 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:10.449615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:10.449938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] PATH_INFO: `/v3/users/0f1585eb50b24adab2ef6febdf06da8f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:10.451133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1585eb50b24adab2ef6febdf06da8f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:10.453396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:10.546894 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1585eb50b24adab2ef6febdf06da8f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.547677 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1585eb50b24adab2ef6febdf06da8f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.548810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1585eb50b24adab2ef6febdf06da8f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.562991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1585eb50b24adab2ef6febdf06da8f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.578327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-83af1530-96ab-43b1-afd1-28f65a4e32fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1585eb50b24adab2ef6febdf06da8f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:10.604523 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1323/2651] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:10 2022] DELETE /identity/v3/users/0f1585eb50b24adab2ef6febdf06da8f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:10.914626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:10.916292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:10.930574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b459-da7f-4179-9a25-ed87089f6dcf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.001483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.002083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.002361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.002631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] PATH_INFO: `/v3/projects/aa76843d608f4117a1de39ad15ebccd2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.003817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa76843d608f4117a1de39ad15ebccd2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.005765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.058280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa76843d608f4117a1de39ad15ebccd2', 'request_id': 'req-0eee2cd4-4d7e-45e5-abda-075197c0e448'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:11.066280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa76843d608f4117a1de39ad15ebccd2', 'request_id': 'req-0eee2cd4-4d7e-45e5-abda-075197c0e448'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:11.073532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa76843d608f4117a1de39ad15ebccd2', 'request_id': 'req-0eee2cd4-4d7e-45e5-abda-075197c0e448'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:11.074936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:11.087203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] The token cache is being invalidate because project aa76843d608f4117a1de39ad15ebccd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:11.087535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:11.088201 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0eee2cd4-4d7e-45e5-abda-075197c0e448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:11.089670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1329/2652] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:10 2022] DELETE /identity/v3/projects/aa76843d608f4117a1de39ad15ebccd2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:11.104454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98882-4150-4206-9eba-31a58f58640b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.104454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98882-4150-4206-9eba-31a58f58640b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.104454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98882-4150-4206-9eba-31a58f58640b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.116745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-71e98882-4150-4206-9eba-31a58f58640b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:11.117991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-71e98882-4150-4206-9eba-31a58f58640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:11.163246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:11.180700 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1324/2653] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:11.189102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.189339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.189613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.212530 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:11.213961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:11.222660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1330/2654] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:11.227469 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98882-4150-4206-9eba-31a58f58640b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.238370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98882-4150-4206-9eba-31a58f58640b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.243706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.244287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.244571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.244848 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.245517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:11.245869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.247799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.259755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1325/2655] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:11.264309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.273564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d4aa-c507-446a-b71f-b418d103556d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.278198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.278825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.279111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.279377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.280002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.281945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.309532 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1331/2656] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:11.313832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.324324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe08-dc1f-481a-b381-c7a38ea87f1e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.387997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.388606 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.388891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.389170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.389860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.391781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.413656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1326/2657] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:11.418204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.428938 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affda583-3bc6-4c57-b845-df9a491c0eb7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.433455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.434030 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.434306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.434575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.435200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.436773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.446731 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1332/2658] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:11.450507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.461114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a402dcf-43b4-47ed-aa25-c8908784a31b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.466165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.466782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.467067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.467347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.467992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.469603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361a01d-42e8-4b56-b642-38f6bd7a5342 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.481083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1327/2659] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:11.485534 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.496141 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b6ee5-57d2-4fe7-a11f-29629bde4cca admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.500673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.501302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.501579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.501893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] PATH_INFO: `/v3/projects/153c2797175646ccb435047fb3b330cb/users/00ece1e229be4174818fc5850084dbdd/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.518328 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153c2797175646ccb435047fb3b330cb, user_id=00ece1e229be4174818fc5850084dbdd, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.521146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.534490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1333/2660] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:11 2022] PUT /identity/v3/projects/153c2797175646ccb435047fb3b330cb/users/00ece1e229be4174818fc5850084dbdd/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:11.547104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.547386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.547657 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.578630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] MFA Rules not processed for user `00ece1e229be4174818fc5850084dbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:11.579832 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:11.645923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1328/2661] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:11.650794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.716469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e76b1-c5b4-43b7-bfa2-d51c2e2b0c01 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.721472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.722119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.722408 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.722742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.723381 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.726011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.754485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1334/2662] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:11.758873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.770534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803d1a7-0398-40df-9053-6231e8703183 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.823124 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:11.840301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.840914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.841226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.841503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.842304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.844366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.868379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1329/2663] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:11 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:11.872431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.882495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0944d0e-a209-4f1c-bb08-3fcf49f6f73e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.887066 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.887641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.887920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.888192 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.889172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.890827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.901560 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1335/2664] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:11.906044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.917209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d546c2-63c9-429c-a133-e5f812f380ba admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:11.921920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:11.922582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:11.922866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:11.923145 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] PATH_INFO: `/v3/projects/47ebd2176c6846d4aac0636f787b98ba/users/98f6b0112c46467381ab2181163193a9/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:11.938231 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ebd2176c6846d4aac0636f787b98ba, user_id=98f6b0112c46467381ab2181163193a9, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:11.940837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:11.953581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1330/2665] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:11 2022] PUT /identity/v3/projects/47ebd2176c6846d4aac0636f787b98ba/users/98f6b0112c46467381ab2181163193a9/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:11.957546 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:11.968443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be42fc-577e-4e1d-bf9b-399332835c15 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:12.024655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:12.025283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:12.025557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:12.025867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:12.026522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:12.028180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec55f3-907e-4c38-88d6-4c736151f9af admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:12.029664 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:12.039469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1336/2666] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:12.043712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:12.054029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873a92b-f855-4dd4-9278-2eb53f3a0813 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:12.058728 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:12.059373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:12.059660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:12.059963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] PATH_INFO: `/v3/projects/47ebd2176c6846d4aac0636f787b98ba/users/98f6b0112c46467381ab2181163193a9/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:12.062044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ebd2176c6846d4aac0636f787b98ba, user_id=98f6b0112c46467381ab2181163193a9, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:12.064716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:12.075558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1331/2667] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:12 2022] PUT /identity/v3/projects/47ebd2176c6846d4aac0636f787b98ba/users/98f6b0112c46467381ab2181163193a9/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:12.086882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:12.087175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:12.087462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:12.115517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] MFA Rules not processed for user `98f6b0112c46467381ab2181163193a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:12.116881 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:12.192517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1337/2668] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:12.205165 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:12.207419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:12.268717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944dac3-037e-4fe7-8be1-5fa8d8ab1d2e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:12.272980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:12.273587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:12.273905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:12.274182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:12.283195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:12.284615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c43c98-678b-41e9-adb9-7f9c9b8e7180 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:12.295104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1332/2669] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:12.913768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:12.977523 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77174c66-fd94-4d7b-bc1e-12660c72b1c8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:12.982458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:12.983094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:12.983383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:12.983662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] PATH_INFO: `/v3/users/00ece1e229be4174818fc5850084dbdd` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:12.984818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ece1e229be4174818fc5850084dbdd)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:12.986781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:13.040372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ece1e229be4174818fc5850084dbdd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.041082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ece1e229be4174818fc5850084dbdd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.041800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ece1e229be4174818fc5850084dbdd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.049828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ece1e229be4174818fc5850084dbdd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.066921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ece1e229be4174818fc5850084dbdd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.078642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:13.083315 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1338/2670] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:12 2022] DELETE /identity/v3/users/00ece1e229be4174818fc5850084dbdd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:13.088151 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:13.088662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:13.088662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:13.108723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:13.109978 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:13.171368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1333/2671] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:13.181297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:13.182768 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:13.242931 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98682027-8b8f-4378-a8b7-fd135cc08baa admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:13.248213 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:13.249046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:13.249524 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:13.249844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:13.261215 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:13.262696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:13.273457 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1339/2672] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:13.572379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:13.574079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:13.589931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02380c-cf5a-4c0c-8998-b6bd6a9c444c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:13.594785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:13.595417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:13.595566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:13.595942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] PATH_INFO: `/v3/projects/153c2797175646ccb435047fb3b330cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:13.596970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=153c2797175646ccb435047fb3b330cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:13.598889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:13.644175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:13.656945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153c2797175646ccb435047fb3b330cb', 'request_id': 'req-b3c4643b-e252-4026-89f5-2dbda4e3aad2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.663461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153c2797175646ccb435047fb3b330cb', 'request_id': 'req-b3c4643b-e252-4026-89f5-2dbda4e3aad2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.669784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153c2797175646ccb435047fb3b330cb', 'request_id': 'req-b3c4643b-e252-4026-89f5-2dbda4e3aad2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.683805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] The token cache is being invalidate because project 153c2797175646ccb435047fb3b330cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:13.684097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.684717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.686299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1334/2673] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:13 2022] DELETE /identity/v3/projects/153c2797175646ccb435047fb3b330cb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:13.690278 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:13.700348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fb74e-f497-4961-9960-c405d4f7da08 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:13.758306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:13.759048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:13.759386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:13.759688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] PATH_INFO: `/v3/users/98f6b0112c46467381ab2181163193a9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:13.760875 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f6b0112c46467381ab2181163193a9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:13.762914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:13.813808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f6b0112c46467381ab2181163193a9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.814461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f6b0112c46467381ab2181163193a9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.815070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f6b0112c46467381ab2181163193a9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.821839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f6b0112c46467381ab2181163193a9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.833640 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cecb0ef7-2078-4033-9ecd-743d21500994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f6b0112c46467381ab2181163193a9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:13.847680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1340/2674] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:13 2022] DELETE /identity/v3/users/98f6b0112c46467381ab2181163193a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:14.135860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.137187 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.152581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4643b-e252-4026-89f5-2dbda4e3aad2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.215450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.216084 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.216363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.216636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] PATH_INFO: `/v3/projects/47ebd2176c6846d4aac0636f787b98ba` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.217833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ebd2176c6846d4aac0636f787b98ba)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.219717 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.276643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ebd2176c6846d4aac0636f787b98ba', 'request_id': 'req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.283074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ebd2176c6846d4aac0636f787b98ba', 'request_id': 'req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.288863 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ebd2176c6846d4aac0636f787b98ba', 'request_id': 'req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.307163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] The token cache is being invalidate because project 47ebd2176c6846d4aac0636f787b98ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:14.307380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.308046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.309532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1335/2675] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:14 2022] DELETE /identity/v3/projects/47ebd2176c6846d4aac0636f787b98ba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:14.314599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed795557-33a7-4153-9a1f-0af0010e5f0c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.315087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed795557-33a7-4153-9a1f-0af0010e5f0c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.315389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed795557-33a7-4153-9a1f-0af0010e5f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.335999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ed795557-33a7-4153-9a1f-0af0010e5f0c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:14.337199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ed795557-33a7-4153-9a1f-0af0010e5f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.356439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:14.435361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.437759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.438024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1341/2676] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:14.440138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.440426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.440705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.452747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb797b7-bf06-4e9a-bf93-01661b10ba9a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.462490 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:14.462683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.463489 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.463718 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.463885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.464569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] PATH_INFO: `/v3/users/7d628776253a457aaea4d5d56430723f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.465339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d628776253a457aaea4d5d56430723f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.468904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.472087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1342/2677] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:14.476809 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.486815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591417a-68b3-4bc8-959f-d2a000e81780 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.493357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.494083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.494478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.494830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.495592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:14.495974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.497816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.517662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1343/2678] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:12:14.518783 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.533544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d628776253a457aaea4d5d56430723f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.534465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d628776253a457aaea4d5d56430723f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.535101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d628776253a457aaea4d5d56430723f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.535812 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bab6e9-306f-43e8-b234-f4fc08b1285d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.538071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:14.543332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.544390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.544850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.545292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.546300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.549135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d628776253a457aaea4d5d56430723f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.549240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.566562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1ab210ce-9df1-480b-8c51-32a0232dd148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d628776253a457aaea4d5d56430723f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:14.585537 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1344/2679] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:14.586655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.586904 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1336/2680] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:14 2022] DELETE /identity/v3/users/7d628776253a457aaea4d5d56430723f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:14.589636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.591475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.591762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.592035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.611936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:14.613126 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.662763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac08eea-57c3-4868-9e5e-e3d5e12d5190 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.673734 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1337/2681] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:14.684478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.686051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.721874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.722493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.722779 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.723062 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.723686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.725663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.747529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc5830-aa5b-4ed3-845a-e71321f6df16 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.751643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.752289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.752390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1345/2682] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:14.753027 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.753308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.754051 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.763445 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.764407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7f34-74fc-4f96-9718-94bd23dc7b33 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.764885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.769236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.769867 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.770160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.770433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.771088 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.772746 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.776036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1338/2683] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:14.783274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1346/2684] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:14.787509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.798347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265b3b-28bf-400a-b018-01b08718eb3b service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.804264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.804884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.805154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.805486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.806189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.807866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94127e87-e60d-4002-b747-00fac6a7887d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.822016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1339/2685] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:14.823058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:14.833690 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c53ca-1af0-49f3-8cc9-d5a42d372460 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:14.839368 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:14.840233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.840515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.840792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] PATH_INFO: `/v3/projects/ecee9b123f654198b744072bfb1d032c/users/7649ae5c698449fdb929d773f5bb1757/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.843236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:14.858835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecee9b123f654198b744072bfb1d032c, user_id=7649ae5c698449fdb929d773f5bb1757, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:14.861477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:14.875379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1347/2686] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:14 2022] PUT /identity/v3/projects/ecee9b123f654198b744072bfb1d032c/users/7649ae5c698449fdb929d773f5bb1757/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:14.887139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:14.887431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:14.887703 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:14.918259 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] MFA Rules not processed for user `7649ae5c698449fdb929d773f5bb1757`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:14.919748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:14.972062 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:14.999930 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1340/2687] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:15.004719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.116347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbdd9-d6af-473e-b853-f67d65fbd3f9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.124054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.124054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.124054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.124054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.124618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.125937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.127685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.128162 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.149135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c359c4-7aba-4bb1-b3d9-cb82ee201dee None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.154322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.154915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.155224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.155505 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] PATH_INFO: `/v3/projects/56772c6e93e9411f90c9b6dca26779cc` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.156650 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56772c6e93e9411f90c9b6dca26779cc)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.159361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.166483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1348/2688] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:15 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:15.170684 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.218565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56772c6e93e9411f90c9b6dca26779cc', 'request_id': 'req-b5909160-a9ee-4483-a525-0ea8eb9aa859'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:15.226723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56772c6e93e9411f90c9b6dca26779cc', 'request_id': 'req-b5909160-a9ee-4483-a525-0ea8eb9aa859'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:15.232696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56772c6e93e9411f90c9b6dca26779cc', 'request_id': 'req-b5909160-a9ee-4483-a525-0ea8eb9aa859'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:15.242694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9366c-1b21-44a4-a3b6-b51fe5d4590d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.247603 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] The token cache is being invalidate because project 56772c6e93e9411f90c9b6dca26779cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:15.247895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:15.248527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:15.250046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1341/2689] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:15 2022] DELETE /identity/v3/projects/56772c6e93e9411f90c9b6dca26779cc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:15.309689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.310340 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.310624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.310900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.311564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.313541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.336647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1349/2690] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:15 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:15.340827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.342415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.359004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5909160-a9ee-4483-a525-0ea8eb9aa859 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.364419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.365193 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.366183 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.366591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.367267 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.369017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.380762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1342/2691] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:15.385300 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.392444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:15.396120 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8d2fd-63bb-4f5e-bb66-7225f1bc931b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.402826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.403887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.404171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.404451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] PATH_INFO: `/v3/projects/e293b7bd75a0440d94d5e20865c9f299/users/a75f6da119dc405388bb7576cf7b903e/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.425363 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e293b7bd75a0440d94d5e20865c9f299, user_id=a75f6da119dc405388bb7576cf7b903e, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.428847 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.443433 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1350/2692] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:15 2022] PUT /identity/v3/projects/e293b7bd75a0440d94d5e20865c9f299/users/a75f6da119dc405388bb7576cf7b903e/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:15.447532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.458578 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d70a-63cb-4b69-994a-458794208c44 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.531534 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.532140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.532423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.532701 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.533342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.534990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e363a0-9fda-4c78-b503-4b2541296a4d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.547112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1343/2693] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:15.551106 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.562133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94c97-cc2c-48e3-ad10-ba96b03c7ff2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.566798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.567478 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.567762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.568041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] PATH_INFO: `/v3/projects/e293b7bd75a0440d94d5e20865c9f299/users/a75f6da119dc405388bb7576cf7b903e/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.569984 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e293b7bd75a0440d94d5e20865c9f299, user_id=a75f6da119dc405388bb7576cf7b903e, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.572669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.583021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1351/2694] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:15 2022] PUT /identity/v3/projects/e293b7bd75a0440d94d5e20865c9f299/users/a75f6da119dc405388bb7576cf7b903e/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:15.593787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.594074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.594364 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.623380 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] MFA Rules not processed for user `a75f6da119dc405388bb7576cf7b903e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:15.624607 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.671077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:15.700663 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1344/2695] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:15.713649 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.715504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.778277 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b95e0-ce9e-49bf-90b6-cbfab9b4a0da admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.782097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.782772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.783121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.783471 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.792512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.793962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.803828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1352/2696] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:15.881871 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.891888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1618c-487b-46fe-895d-94a3e8d804ec None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.895586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.896197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.896480 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.897327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.899107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.906075 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:15.907451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:15.963586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.965044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609a700-20f9-43e3-9959-e79eaf0ef06f service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:15.974248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1345/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:15.976846 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37a3e-e403-48d2-a434-eb97fd8f8b3a service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:15.983050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:15.983707 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:15.983992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:15.984276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] PATH_INFO: `/v3/users/3eded341c7864cefa2db3a6c3077da4e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:15.985495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eded341c7864cefa2db3a6c3077da4e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:15.987692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.041432 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eded341c7864cefa2db3a6c3077da4e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.042528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eded341c7864cefa2db3a6c3077da4e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.043164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eded341c7864cefa2db3a6c3077da4e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.050626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eded341c7864cefa2db3a6c3077da4e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.063390 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:16.064680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eded341c7864cefa2db3a6c3077da4e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.087971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1353/2698] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:15 2022] DELETE /identity/v3/users/3eded341c7864cefa2db3a6c3077da4e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:16.094417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.094790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.095135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.116108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:16.117248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.187500 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1346/2699] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:16.197109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.198712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.260880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a472c-0234-4c24-934b-220753b9b5c7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.264844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:16.265583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.265981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.266326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.276281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.278204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.278361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:16.279348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b36c6-3e46-4e3a-8d2f-158f7ef3cde7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.290122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806da53-1b55-4b2f-9c40-5488a180f9c5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.290426 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1354/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:16.294794 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:16.295408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.295687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.295959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] PATH_INFO: `/v3/users/7649ae5c698449fdb929d773f5bb1757` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.297077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7649ae5c698449fdb929d773f5bb1757)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:16.299019 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.355345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7649ae5c698449fdb929d773f5bb1757'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.356139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7649ae5c698449fdb929d773f5bb1757'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.356901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7649ae5c698449fdb929d773f5bb1757'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.374658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7649ae5c698449fdb929d773f5bb1757'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.389800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7649ae5c698449fdb929d773f5bb1757'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.391660 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:16.405143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1347/2701] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:16 2022] DELETE /identity/v3/users/7649ae5c698449fdb929d773f5bb1757 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:16.409439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.409932 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.410006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.429553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:16.430890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.513595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1355/2702] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:16.523453 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.525583 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.573933 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.575367 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.589396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7218e6d-592a-47d1-9b3b-d133d138f456 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.594281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:16.594935 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.595221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.595496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] PATH_INFO: `/v3/projects/5521770de7e8440092bff9cca7460eea` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.596697 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5521770de7e8440092bff9cca7460eea)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:16.598698 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.607574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d697b46-247a-438f-88bb-a0c003bbb01c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.611799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:16.612381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.612642 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.612914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.622353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:16.623854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.634560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1348/2703] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:16.651631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5521770de7e8440092bff9cca7460eea', 'request_id': 'req-747a1c3c-e757-400f-9fe9-297f86f6ddf3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.660816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5521770de7e8440092bff9cca7460eea', 'request_id': 'req-747a1c3c-e757-400f-9fe9-297f86f6ddf3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.667461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5521770de7e8440092bff9cca7460eea', 'request_id': 'req-747a1c3c-e757-400f-9fe9-297f86f6ddf3'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.678488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] The token cache is being invalidate because project 5521770de7e8440092bff9cca7460eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:16.678795 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.679464 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.681214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1356/2704] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:16 2022] DELETE /identity/v3/projects/5521770de7e8440092bff9cca7460eea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:16.691851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.707748 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff276b-ff4c-4843-b4e6-5d11fa2897df service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.775051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:16.775690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:16.776195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:16.776341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] PATH_INFO: `/v3/users/5d8c962b66e34df0b693467c5b4220d8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:16.777711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8c962b66e34df0b693467c5b4220d8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:16.779670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:16.838707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8c962b66e34df0b693467c5b4220d8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.839386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8c962b66e34df0b693467c5b4220d8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.840025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8c962b66e34df0b693467c5b4220d8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.847821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8c962b66e34df0b693467c5b4220d8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.864253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8c962b66e34df0b693467c5b4220d8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:16.879436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1349/2705] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:16 2022] DELETE /identity/v3/users/5d8c962b66e34df0b693467c5b4220d8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:16.940889 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:16.942511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:16.966852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a1c3c-e757-400f-9fe9-297f86f6ddf3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:16.983004 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:17.033007 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.033650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.033982 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.034260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] PATH_INFO: `/v3/projects/ecee9b123f654198b744072bfb1d032c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.035384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecee9b123f654198b744072bfb1d032c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.037327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.093483 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecee9b123f654198b744072bfb1d032c', 'request_id': 'req-e9c62915-89ca-494b-afbd-47255561de5d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.101447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecee9b123f654198b744072bfb1d032c', 'request_id': 'req-e9c62915-89ca-494b-afbd-47255561de5d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.109772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecee9b123f654198b744072bfb1d032c', 'request_id': 'req-e9c62915-89ca-494b-afbd-47255561de5d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.127460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] The token cache is being invalidate because project ecee9b123f654198b744072bfb1d032c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:17.127839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.128577 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.130612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1357/2706] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:16 2022] DELETE /identity/v3/projects/ecee9b123f654198b744072bfb1d032c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:17.134990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.136414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.150155 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce878-9c89-4969-bed0-1dd6419f9963 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.160680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:17.197952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.199634 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.228386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.229049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.229301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.229602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] PATH_INFO: `/v3/users/a75f6da119dc405388bb7576cf7b903e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.230851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75f6da119dc405388bb7576cf7b903e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.233083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.276452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c62915-89ca-494b-afbd-47255561de5d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.281773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.282479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.282883 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.283155 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] PATH_INFO: `/v3/projects/66263caab08d4a19900b4276432b003b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.284269 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66263caab08d4a19900b4276432b003b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.286235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.286888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75f6da119dc405388bb7576cf7b903e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.288074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75f6da119dc405388bb7576cf7b903e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.288674 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75f6da119dc405388bb7576cf7b903e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.296494 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75f6da119dc405388bb7576cf7b903e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.309661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-00ecec2f-b274-4c0e-8584-b1cc088a6a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75f6da119dc405388bb7576cf7b903e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.322716 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1350/2707] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:17 2022] DELETE /identity/v3/users/a75f6da119dc405388bb7576cf7b903e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:17.339114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:17.341179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66263caab08d4a19900b4276432b003b', 'request_id': 'req-611f2ed7-ad3b-44b9-8094-874bf5bec147'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.352259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66263caab08d4a19900b4276432b003b', 'request_id': 'req-611f2ed7-ad3b-44b9-8094-874bf5bec147'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.358628 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66263caab08d4a19900b4276432b003b', 'request_id': 'req-611f2ed7-ad3b-44b9-8094-874bf5bec147'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.380268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] The token cache is being invalidate because project 66263caab08d4a19900b4276432b003b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:17.380526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.381360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-611f2ed7-ad3b-44b9-8094-874bf5bec147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.383087 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1358/2708] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:17 2022] DELETE /identity/v3/projects/66263caab08d4a19900b4276432b003b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:17.387811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.388321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.388598 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.408499 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:17.409866 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.474470 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1351/2709] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:17.483079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.483495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.483769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.509355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:17.510621 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.518747 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1359/2710] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:17.523082 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.532984 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f97b6-172c-4962-8c93-f0b55a08d102 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.538509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.539152 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.539438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.539718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.540423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:17.540735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.542381 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.555698 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1352/2711] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:17.560002 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.569991 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9239-c905-4773-b50d-26af13f2d657 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.574890 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.575511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.575801 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.576074 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.576728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.578962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.612515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1360/2712] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:17.613666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.624715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ea6d-6baf-4411-aaa8-36ce5845762c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.642929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:17.694418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.695046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.695309 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.695638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.696285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.698221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.720816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1353/2713] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:17.725439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.736001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713dc1a-20b4-4c9e-86e1-58df6da0a569 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.740742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.741352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.741635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.741943 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.742585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.744185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.754275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1361/2714] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:17.758346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.768472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2501082-7ed5-44d7-a8ca-0b3a33c9d736 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.773590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.774218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.774503 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.774805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.775513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.777065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.789686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1354/2715] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:17.793686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.804924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b01ce-8ba0-47a7-865f-5f19227565e1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.812679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.813407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.813841 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.814142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] PATH_INFO: `/v3/projects/816623d2a39048238fb1839ff52f33ed/users/4fd5a0a59b5c4c3a8f7492dfa22a16c5/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.830355 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.832179 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.832920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] RBAC: Authorizing `identity:create_grant(project_id=816623d2a39048238fb1839ff52f33ed, user_id=4fd5a0a59b5c4c3a8f7492dfa22a16c5, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.836711 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf091d-1f40-458d-ac04-22c8fd78826b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.848600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac26ce-74c0-4ee6-bde8-68cf8b515577 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:17.854458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:17.855298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.855676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.856025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] PATH_INFO: `/v3/projects/e293b7bd75a0440d94d5e20865c9f299` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.857332 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e293b7bd75a0440d94d5e20865c9f299)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:17.859417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:17.873182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1362/2716] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:17 2022] PUT /identity/v3/projects/816623d2a39048238fb1839ff52f33ed/users/4fd5a0a59b5c4c3a8f7492dfa22a16c5/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:17.883032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.883316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.883604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.909283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e293b7bd75a0440d94d5e20865c9f299', 'request_id': 'req-2e85b170-57aa-421d-992d-6fa0a91076d4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.911037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] MFA Rules not processed for user `4fd5a0a59b5c4c3a8f7492dfa22a16c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:17.912356 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.916047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e293b7bd75a0440d94d5e20865c9f299', 'request_id': 'req-2e85b170-57aa-421d-992d-6fa0a91076d4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.921923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e293b7bd75a0440d94d5e20865c9f299', 'request_id': 'req-2e85b170-57aa-421d-992d-6fa0a91076d4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.934426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] The token cache is being invalidate because project e293b7bd75a0440d94d5e20865c9f299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:17.934721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.935353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2e85b170-57aa-421d-992d-6fa0a91076d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:17.940164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1355/2717] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:17 2022] DELETE /identity/v3/projects/e293b7bd75a0440d94d5e20865c9f299 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:17.941602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61246-16f3-4c94-8740-924b3e68ead7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:17.941914 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61246-16f3-4c94-8740-924b3e68ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:17.942194 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61246-16f3-4c94-8740-924b3e68ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:17.962338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7aa61246-16f3-4c94-8740-924b3e68ead7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:17.963551 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7aa61246-16f3-4c94-8740-924b3e68ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:17.978198 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1363/2718] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:17.982756 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:17.984063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.025324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1356/2719] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:18.033989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.034282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.034566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.048895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d0c81-757b-4ed1-be20-8ae9fc595e0b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.053625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.053968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:18.054260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.054541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.054815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.055173 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.055441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.057330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.062553 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.063487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1357/2720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:18.067474 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.077440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e5e5-3fb8-4c23-90c2-8c55c4c2f9d7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.082790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.083412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.083692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.083964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.085170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:18.085477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.087128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.087552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1364/2721] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:17 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:18.091614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.098942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1358/2722] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:12:18.103065 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.107856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.114745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c28ef-7b25-40e2-a835-b9719cb56d9d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.154132 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48675b-f7b9-4642-addf-f85d153c03a9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.158930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.159552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.159796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.160079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.160700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.162725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.174560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.175174 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.175458 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.175735 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.176369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.178347 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.190435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1365/2723] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:18.192028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.203721 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aefc0-7842-4606-ba42-256537e125ae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.210085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.210691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.211197 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.211317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1359/2724] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:18.211427 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.212014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.213609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.215188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.225048 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1366/2725] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:18.228934 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.241202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed72-a94e-4b25-9594-d357245e024b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.285052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d49d8-6c4d-4083-84e6-8c0acbf36862 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.289797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.290410 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.290715 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.290988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.291618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.293527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.299517 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.300175 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.300461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.300742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] PATH_INFO: `/v3/projects/8204a3e0ec8f4de0a6e31b7938573640/users/738b671c3bce41acb3eaf7cc254367de/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.318088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1360/2726] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/users => generated 280 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:18.318257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8204a3e0ec8f4de0a6e31b7938573640, user_id=738b671c3bce41acb3eaf7cc254367de, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.319662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.320347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.330436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197774af-840b-4eb7-8424-40538826be80 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.333669 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1367/2727] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:18 2022] PUT /identity/v3/projects/8204a3e0ec8f4de0a6e31b7938573640/users/738b671c3bce41acb3eaf7cc254367de/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:18.338322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.387311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.394936 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.395670 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.396058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.396438 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.397185 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.398926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.401557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe9eba-a9ec-4c66-929b-4e2587e69823 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.406238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.406865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.407156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.407428 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.408059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.409457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1361/2728] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:18.409719 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.413330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.419620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1368/2729] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:18.422905 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5df28-48fe-430a-83b6-38abae3171d5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.423359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.427483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.428634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.428920 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.429195 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.429867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.431442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.433345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884172ed-cbb2-4a2b-be0f-8ea489c2f421 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.438032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.438804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.438979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.439262 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] PATH_INFO: `/v3/projects/8204a3e0ec8f4de0a6e31b7938573640/users/738b671c3bce41acb3eaf7cc254367de/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.441171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8204a3e0ec8f4de0a6e31b7938573640, user_id=738b671c3bce41acb3eaf7cc254367de, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.441766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1362/2730] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:18.446334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.447570 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa584c-df78-48f3-9dce-d5e681ca7020 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.451036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.456625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8c100-2d09-4f47-95d7-cfcf832026de admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.459526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1369/2731] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:18 2022] PUT /identity/v3/projects/8204a3e0ec8f4de0a6e31b7938573640/users/738b671c3bce41acb3eaf7cc254367de/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:18.471029 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.471316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.471591 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.499670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] MFA Rules not processed for user `738b671c3bce41acb3eaf7cc254367de`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:18.500887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.517097 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.517769 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.518052 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.518323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] PATH_INFO: `/v3/projects/276d20113cf8470c81e442f5cfddf846/users/a23abbcf0e9b452db2da49d15491f864/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.532868 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=276d20113cf8470c81e442f5cfddf846, user_id=a23abbcf0e9b452db2da49d15491f864, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.535533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68787a29-b51d-45ba-bc30-f8a290d88b64 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.548490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1363/2732] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:18 2022] PUT /identity/v3/projects/276d20113cf8470c81e442f5cfddf846/users/a23abbcf0e9b452db2da49d15491f864/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:18.559548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.559836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.560110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.572292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1370/2733] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:18.584137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.585650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.588323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] MFA Rules not processed for user `a23abbcf0e9b452db2da49d15491f864`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:18.589535 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.643514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdccd1-d1cb-48a0-821b-90dc4064bd95 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.655449 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1364/2734] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:18.689539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.690971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:18.702999 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.703673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.703960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.704241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.732550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.751192 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38f786-a696-4445-bae6-caddf34dbb67 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.755221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.755869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.756138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.756412 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.764638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.766242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.767703 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.769129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.775379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1365/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:18.778738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1371/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:18.791456 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.812444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:18.858796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:18.864635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5740ae-2677-4c33-864a-8a1a8bc042f0 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.867870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d953-31cd-4517-a84d-4be8a44cf810 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:18.870157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.870820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.871088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.871366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] PATH_INFO: `/v3/users/a23abbcf0e9b452db2da49d15491f864` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.871646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:18.872243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.872509 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23abbcf0e9b452db2da49d15491f864)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.872610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.872902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.874579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.921232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23abbcf0e9b452db2da49d15491f864'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:18.921869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23abbcf0e9b452db2da49d15491f864'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:18.922537 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23abbcf0e9b452db2da49d15491f864'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:18.930658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23abbcf0e9b452db2da49d15491f864'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:18.932988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:18.934686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b0b6f-8288-4f77-b976-3f2b5cf61555 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:18.943507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23abbcf0e9b452db2da49d15491f864'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:18.943635 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1372/2737] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:18.958272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1366/2738] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:18 2022] DELETE /identity/v3/users/a23abbcf0e9b452db2da49d15491f864 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:18.963164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:18.963508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:18.963790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:18.983326 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:18.984733 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:19.051797 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1373/2739] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:19.061790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.063206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:19.123763 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87616ecd-b4c3-4b9e-b7dd-8bfa637d4ecb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.127661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.128261 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.128601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.128874 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.138180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.139599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23808fa9-21da-4e4d-97eb-6476014b67ec service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.149404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1367/2740] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:19.422574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.424022 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:19.438235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8fea3-0ef7-48a7-a569-6c26d5806e2a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.444050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.444713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.444992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.445273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] PATH_INFO: `/v3/projects/276d20113cf8470c81e442f5cfddf846` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.446425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=276d20113cf8470c81e442f5cfddf846)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.448324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.495660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276d20113cf8470c81e442f5cfddf846', 'request_id': 'req-abec23a9-9009-43fc-b61a-5b0beb118045'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:19.502304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276d20113cf8470c81e442f5cfddf846', 'request_id': 'req-abec23a9-9009-43fc-b61a-5b0beb118045'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:19.503475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:19.510896 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276d20113cf8470c81e442f5cfddf846', 'request_id': 'req-abec23a9-9009-43fc-b61a-5b0beb118045'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:19.521528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] The token cache is being invalidate because project 276d20113cf8470c81e442f5cfddf846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:19.522434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:19.523065 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-abec23a9-9009-43fc-b61a-5b0beb118045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:19.524535 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1374/2741] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:19 2022] DELETE /identity/v3/projects/276d20113cf8470c81e442f5cfddf846 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:19.529348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.529797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.530058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.551059 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:19.552262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:19.587066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:19.614159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1368/2742] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:19.621968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.622259 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.622566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.642353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:19.643557 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:19.651983 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1375/2743] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:19.656022 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.665465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7dff6-ff14-4a65-8f40-46e494f85510 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.670046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.670634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.670916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.671188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.671824 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:19.672127 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.673720 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.684926 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1369/2744] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:19.688947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.698484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33050f70-d574-4eae-aea3-89a3af386afe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.703394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.703979 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.704260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.704533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.705160 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.707084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.735118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1376/2745] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:19 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:19.739323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.749128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15878d-7690-4c1a-ae07-146d7ef95108 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.804667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.805265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.805549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.805861 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.806484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.808389 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.829759 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1370/2746] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:19 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:19.834766 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.844868 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e6288-a835-4f7a-af9c-8cad2535d901 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.849646 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.850297 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.850572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.850844 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.851502 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.853079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.862961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1377/2747] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:19.867353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.877370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9090c-cdd8-4b2a-aa7e-64871852fe93 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.882691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.883287 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.883567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.883913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.884831 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.886638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad868fda-6b3c-48d7-82a5-e6b5c86a78a0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.897374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1371/2748] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:19.901301 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:19.910968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e6fb4-a951-40c5-9950-bcfc12cb50cc admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:19.915496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:19.916157 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.916434 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.916708 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] PATH_INFO: `/v3/projects/82ff53810f744f7d96126d1eed8662c5/users/7af84fe0a9534f2da43e238124c430db/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.931308 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ff53810f744f7d96126d1eed8662c5, user_id=7af84fe0a9534f2da43e238124c430db, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:19.933954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:19.947254 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1378/2749] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:19 2022] PUT /identity/v3/projects/82ff53810f744f7d96126d1eed8662c5/users/7af84fe0a9534f2da43e238124c430db/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:19.952344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a84e7-7f81-4fb8-a1cb-475631ca21c1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:19.952634 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a84e7-7f81-4fb8-a1cb-475631ca21c1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:19.952915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a84e7-7f81-4fb8-a1cb-475631ca21c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:19.971477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c21a84e7-7f81-4fb8-a1cb-475631ca21c1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:19.972662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c21a84e7-7f81-4fb8-a1cb-475631ca21c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:20.031417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1372/2750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:20.040823 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:20.042595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:20.098430 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08666b9f-5ce0-4dc5-b8d9-4e28cf814637 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:20.102170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:20.102780 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:20.103059 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:20.103333 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:20.118046 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:20.119469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:20.129236 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1379/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:20.792476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c08c7-d6a4-4ae8-bdc3-1fda03ece89f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:20.792950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c08c7-d6a4-4ae8-bdc3-1fda03ece89f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:20.793035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c08c7-d6a4-4ae8-bdc3-1fda03ece89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:20.820878 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-314c08c7-d6a4-4ae8-bdc3-1fda03ece89f None None] MFA Rules not processed for user `7af84fe0a9534f2da43e238124c430db`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:20.822273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-314c08c7-d6a4-4ae8-bdc3-1fda03ece89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:20.858056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:20.890424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1373/2752] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:20.902107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:20.903531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:20.962798 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ed281-8385-431a-bd6c-4a5dd76348b1 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:20.966601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:20.967210 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:20.967494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:20.967771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:20.977895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:20.979332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:20.988182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1380/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:22.336229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c209-53a5-4873-b469-f512b862e026 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:22.336696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c209-53a5-4873-b469-f512b862e026 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:22.336803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c209-53a5-4873-b469-f512b862e026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:22.355923 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b708c209-53a5-4873-b469-f512b862e026 None None] MFA Rules not processed for user `7af84fe0a9534f2da43e238124c430db`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:22.357128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b708c209-53a5-4873-b469-f512b862e026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:22.363945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1374/2754] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:22.604437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:22.605910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:22.617782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f6ae2-3204-47e2-be42-852da1dff2dd service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:22.621463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:22.622080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:22.622362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:22.622631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:22.624310 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:22.633681 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:22.635109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:22.642828 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1381/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:22.970418 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c209-53a5-4873-b469-f512b862e026 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:22.971854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b708c209-53a5-4873-b469-f512b862e026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:23.044218 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c209-53a5-4873-b469-f512b862e026 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:23.049518 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:23.050265 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:23.050550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:23.050821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:23.052685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:23.065058 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:23.066658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:23.077209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1375/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:27.243954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:27.252668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c39f1-e061-407c-a4d7-ddb8aa1fc943 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:27.256411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:27.257023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:27.257302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:27.257575 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:27.259306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:27.268977 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:27.270487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857eb13d-9c2c-4484-a0bc-f2797adde09b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:27.274667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:27.279996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1382/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:29.320772 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:29.322731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:29.337753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f08201-dcf2-4f0f-b554-2072b35538b5 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:29.342370 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:29.343011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:29.343299 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:29.343579 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] PATH_INFO: `/v3/users/4fd5a0a59b5c4c3a8f7492dfa22a16c5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:29.344722 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd5a0a59b5c4c3a8f7492dfa22a16c5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:29.346685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:29.398302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd5a0a59b5c4c3a8f7492dfa22a16c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:29.399021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd5a0a59b5c4c3a8f7492dfa22a16c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:29.399635 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd5a0a59b5c4c3a8f7492dfa22a16c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:29.407547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd5a0a59b5c4c3a8f7492dfa22a16c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:29.422294 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd5a0a59b5c4c3a8f7492dfa22a16c5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:29.442454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1376/2758] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:29 2022] DELETE /identity/v3/users/4fd5a0a59b5c4c3a8f7492dfa22a16c5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:29.444218 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:29.444696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:29.444825 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:29.470923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:29.472318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:29.540187 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1383/2759] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:29.547164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:29.548591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:29.584252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:29.611426 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb16a1-9e7b-4a2e-89ac-a6cb98bdf462 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:29.615397 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:29.616016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:29.616298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:29.616572 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:29.626177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:29.627612 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:29.637025 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1377/2760] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:30.140803 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:30.143377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:30.158331 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50d27a-843e-4e7c-80f6-47d890434174 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:30.163866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:30.164680 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:30.165097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:30.165525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] PATH_INFO: `/v3/projects/816623d2a39048238fb1839ff52f33ed` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:30.167043 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816623d2a39048238fb1839ff52f33ed)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:30.169451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:30.266871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816623d2a39048238fb1839ff52f33ed', 'request_id': 'req-af3757c7-bce0-404f-87cf-88dc4e028ad6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.283136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816623d2a39048238fb1839ff52f33ed', 'request_id': 'req-af3757c7-bce0-404f-87cf-88dc4e028ad6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.295720 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816623d2a39048238fb1839ff52f33ed', 'request_id': 'req-af3757c7-bce0-404f-87cf-88dc4e028ad6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.325109 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] The token cache is being invalidate because project 816623d2a39048238fb1839ff52f33ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:30.325418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.326453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.331384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1384/2761] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:30 2022] DELETE /identity/v3/projects/816623d2a39048238fb1839ff52f33ed => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:30.336524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:30.348109 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45733372-0b27-43a7-9868-ba409ed1a9fa service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:30.434159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:30.435051 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:30.435398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:30.436137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] PATH_INFO: `/v3/users/738b671c3bce41acb3eaf7cc254367de` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:30.437669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738b671c3bce41acb3eaf7cc254367de)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:30.440556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:30.558771 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738b671c3bce41acb3eaf7cc254367de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.559387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738b671c3bce41acb3eaf7cc254367de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.560214 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738b671c3bce41acb3eaf7cc254367de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.579738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738b671c3bce41acb3eaf7cc254367de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.605725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-21ec960d-a64f-4e1b-ad8a-03b3255c3db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738b671c3bce41acb3eaf7cc254367de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:30.641688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1378/2762] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:30 2022] DELETE /identity/v3/users/738b671c3bce41acb3eaf7cc254367de => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:31.025500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.027533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:31.048928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3757c7-bce0-404f-87cf-88dc4e028ad6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.111159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.112079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.112444 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.112790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] PATH_INFO: `/v3/projects/8204a3e0ec8f4de0a6e31b7938573640` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.114060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8204a3e0ec8f4de0a6e31b7938573640)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.116651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.151854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:31.180601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8204a3e0ec8f4de0a6e31b7938573640', 'request_id': 'req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:31.192212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8204a3e0ec8f4de0a6e31b7938573640', 'request_id': 'req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:31.203146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8204a3e0ec8f4de0a6e31b7938573640', 'request_id': 'req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:31.215158 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] The token cache is being invalidate because project 8204a3e0ec8f4de0a6e31b7938573640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:31.215451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:31.216100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fdd463b2-4b0b-4dbf-8e73-f97583e165e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:31.217610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1385/2763] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:31 2022] DELETE /identity/v3/projects/8204a3e0ec8f4de0a6e31b7938573640 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:31.222395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.222756 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.223039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.244482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:31.245645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:31.308167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:31.323820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1379/2764] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:31.331849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.332200 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.332422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.354404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:31.355626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:31.364484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1386/2765] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:31.369306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.379446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533108f3-3f34-404d-9a9e-0b06b852e486 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.384487 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.385100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.385386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.385661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.386361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:31.386669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.388281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.400919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1380/2766] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:31.405383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.415347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee699bf-8df0-4d02-815e-fe5332710aca None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.420168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.420774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.421053 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.421882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.422519 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.424514 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.457452 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1387/2767] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:31.458608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.468946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b414a9e-cc9e-4fd3-9af8-23471964cd69 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.541850 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.542532 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.542877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.543302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.544931 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.547171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.577441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1381/2768] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:12:31.578888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.589273 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceea46a-e772-42d4-a4b0-ed763e92ecd9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.593944 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.594612 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.595613 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.595966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.596668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.598385 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.612566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1388/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:31.614246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.624852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a5dee-31b4-4b7f-bd82-8898cb834cbc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.629490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.630125 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.630398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.630672 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.631306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.633446 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc2e9-1c36-4af6-8535-01cc109e27e0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.644336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1382/2770] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:31.649084 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.659406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cda25-9b66-44a5-b669-1bc07a3fff8e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.665864 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.666949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.667233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.667510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] PATH_INFO: `/v3/projects/965f106eabc94150bf0babd57fe82a74/users/16d2ee093f424f29926a362d6602b9fb/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.683274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=965f106eabc94150bf0babd57fe82a74, user_id=16d2ee093f424f29926a362d6602b9fb, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.686014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.700636 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1389/2771] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:31 2022] PUT /identity/v3/projects/965f106eabc94150bf0babd57fe82a74/users/16d2ee093f424f29926a362d6602b9fb/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:31.712833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.713215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.713591 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.741836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] MFA Rules not processed for user `16d2ee093f424f29926a362d6602b9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:31.743197 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:31.813044 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1383/2772] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:31.817316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.881547 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea541b2-8dde-4f66-bfe4-b2664575d90f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:31.891473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:31.892097 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:31.892383 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:31.892760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:31.893417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:31.897608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:31.935750 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1390/2773] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:31.940190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:31.952983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c0ab2-d6dd-4d98-9588-956cf04c5629 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.008257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:32.021123 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.021884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.022164 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.022468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.023094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.025015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.046801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1384/2774] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:31 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:32.050527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.061260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc6732-1e6b-4eb8-b73a-267dc1dba467 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.066563 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.067162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.067441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.067713 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.068350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.070552 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.081364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1391/2775] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:32.085527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.095928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871681cf-2968-4b19-8bbf-7e0d22c76d89 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.100754 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.101455 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.101733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.102009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] PATH_INFO: `/v3/projects/5318b234be024d75aeb456d99b220d19/users/6c44e16699cd4e7eb9beeefabc5bf94d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.117461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5318b234be024d75aeb456d99b220d19, user_id=6c44e16699cd4e7eb9beeefabc5bf94d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.120170 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.134110 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1385/2776] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:32 2022] PUT /identity/v3/projects/5318b234be024d75aeb456d99b220d19/users/6c44e16699cd4e7eb9beeefabc5bf94d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:32.138616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.148568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458f0d-3a94-4411-b394-6de790a6f374 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.168662 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:32.207802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.208572 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.208961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.209321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.210000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.211609 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52634365-ced1-4fec-b369-cadc28c205ec admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.223995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1392/2777] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:32.229932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.243652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8cb1a-b156-4379-a6a9-259fbc85e253 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.248753 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.249539 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.249919 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.250275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] PATH_INFO: `/v3/projects/5318b234be024d75aeb456d99b220d19/users/6c44e16699cd4e7eb9beeefabc5bf94d/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.252496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5318b234be024d75aeb456d99b220d19, user_id=6c44e16699cd4e7eb9beeefabc5bf94d, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.256068 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.266901 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1386/2778] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:32 2022] PUT /identity/v3/projects/5318b234be024d75aeb456d99b220d19/users/6c44e16699cd4e7eb9beeefabc5bf94d/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:32.276995 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.277362 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.277738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.309970 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] MFA Rules not processed for user `6c44e16699cd4e7eb9beeefabc5bf94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:32.311312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:32.401325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1393/2779] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:32.412394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.414485 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:32.425396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.428079 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:32.483870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be79d-6084-4152-aa36-280989729542 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.488061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.488749 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.489038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.489316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.499420 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.500896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.509099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb67de-ff56-47e0-8d28-85c015611b90 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.511137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1387/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:32.514840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.515451 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.515832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.516003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.517764 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:32.531138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.532821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.552339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1394/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:32.598321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:32.607538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603ff42-0320-4fcf-88a4-1207bc545bcd service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:32.611429 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:32.612032 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:32.612343 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:32.612626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:32.681820 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:32.683247 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc00205-e427-41a5-8c37-c0e333318c03 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:32.705234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1388/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:35.479461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:35.513607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:35.543420 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9a27-ad5b-496a-a53c-fbcf5c3288f9 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:35.548018 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:35.548642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:35.548926 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:35.549206 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] PATH_INFO: `/v3/users/16d2ee093f424f29926a362d6602b9fb` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:35.550352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d2ee093f424f29926a362d6602b9fb)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:35.552265 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:35.601588 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d2ee093f424f29926a362d6602b9fb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:35.602360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d2ee093f424f29926a362d6602b9fb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:35.603045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d2ee093f424f29926a362d6602b9fb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:35.611832 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d2ee093f424f29926a362d6602b9fb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:35.626486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d2ee093f424f29926a362d6602b9fb'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:35.641280 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1395/2783] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:35 2022] DELETE /identity/v3/users/16d2ee093f424f29926a362d6602b9fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:35.645800 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:35.646252 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:35.646437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:35.666543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:35.667792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:35.707587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:35.730432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1389/2784] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:35.739850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:35.741820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:35.800431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d7230-6fe1-4793-9ec9-911e590f807a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:35.804625 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:35.805256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:35.805541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:35.805852 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:35.814871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:35.816312 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:35.826516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1396/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:36.090927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:36.092379 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:36.107176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f4e50-7a44-4e96-942c-42cdcb639322 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:36.112016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:36.112854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.113283 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.113555 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] PATH_INFO: `/v3/projects/965f106eabc94150bf0babd57fe82a74` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.114740 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=965f106eabc94150bf0babd57fe82a74)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:36.117308 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:36.163967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965f106eabc94150bf0babd57fe82a74', 'request_id': 'req-159a6a2e-6471-4312-87de-65179c8e72fb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.169990 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965f106eabc94150bf0babd57fe82a74', 'request_id': 'req-159a6a2e-6471-4312-87de-65179c8e72fb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.175450 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965f106eabc94150bf0babd57fe82a74', 'request_id': 'req-159a6a2e-6471-4312-87de-65179c8e72fb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.187220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] The token cache is being invalidate because project 965f106eabc94150bf0babd57fe82a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:36.187529 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.188171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.189625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1390/2786] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:36 2022] DELETE /identity/v3/projects/965f106eabc94150bf0babd57fe82a74 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:36.193590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:36.204250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ed2f7-5b9a-4414-ba65-9f8e7f250bb7 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:36.252683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:36.263454 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:36.264276 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.264586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.264860 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] PATH_INFO: `/v3/users/6c44e16699cd4e7eb9beeefabc5bf94d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.266020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c44e16699cd4e7eb9beeefabc5bf94d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:36.267965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:36.313659 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c44e16699cd4e7eb9beeefabc5bf94d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.314279 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c44e16699cd4e7eb9beeefabc5bf94d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.314887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c44e16699cd4e7eb9beeefabc5bf94d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.322260 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c44e16699cd4e7eb9beeefabc5bf94d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.334027 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1d49a3b4-35b1-4899-9f86-7509c1e6c082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c44e16699cd4e7eb9beeefabc5bf94d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.347672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1397/2787] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:36 2022] DELETE /identity/v3/users/6c44e16699cd4e7eb9beeefabc5bf94d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:36.614668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:36.616699 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:36.630964 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a6a2e-6471-4312-87de-65179c8e72fb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:36.695682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:36.696322 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.696599 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.696876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] PATH_INFO: `/v3/projects/5318b234be024d75aeb456d99b220d19` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.697993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5318b234be024d75aeb456d99b220d19)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:36.699884 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:36.745163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5318b234be024d75aeb456d99b220d19', 'request_id': 'req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.751704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5318b234be024d75aeb456d99b220d19', 'request_id': 'req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.757424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5318b234be024d75aeb456d99b220d19', 'request_id': 'req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.760975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:36.770791 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] The token cache is being invalidate because project 5318b234be024d75aeb456d99b220d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:36.771246 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.772047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5c460f38-b7f2-436d-ba1d-bf0fee00bd24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:36.773576 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1391/2788] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:36 2022] DELETE /identity/v3/projects/5318b234be024d75aeb456d99b220d19 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:36.778371 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.778826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.778910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.798614 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:36.799811 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:36.862630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1398/2789] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:12:36.870568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.870857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.871131 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.891181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:36.892396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:36.901074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1392/2790] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:36.905275 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:36.915169 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca6b43-6ac5-49cd-a781-e3df1c1c16d8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:36.919920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:36.920520 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.920800 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.921078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.921740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:36.922042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:36.923637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:36.935821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1399/2791] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:12:36.939908 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:36.949649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a8ef2-846e-4b80-a9e7-f47a749f05c0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:36.958325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:36.958928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:36.959207 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:36.959477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:36.960181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:36.962654 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:36.992490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1393/2792] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:36 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:36.996650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.007422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eb264-0aa8-434e-95ad-ddde0ad39088 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.065757 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.066369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.066678 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.066954 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.067595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.070057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.078370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:37.092679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1400/2793] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:36 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:37.096751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.107128 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380ed49-70be-464d-ba77-92cc57d90d30 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.111732 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.112336 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.112615 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.112888 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.113526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.115157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.126049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1394/2794] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:37.130090 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.140687 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cf336-1b4e-464d-8044-0d48ea37f94e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.145564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.146034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.146353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.146554 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.147190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.148818 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77793809-ec41-4515-b54a-791ac1716757 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.159587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1401/2795] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:37.163723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.174857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6943783-db38-4cd5-8ba1-141cef0bfd1e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.180081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.180742 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.181028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.181359 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] PATH_INFO: `/v3/projects/a09a05f6a6ba48c898ea6bd5b514d596/users/2f5c72447db84ae8818efb70ccd0c1af/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.196933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09a05f6a6ba48c898ea6bd5b514d596, user_id=2f5c72447db84ae8818efb70ccd0c1af, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.199641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.212961 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1395/2796] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:37 2022] PUT /identity/v3/projects/a09a05f6a6ba48c898ea6bd5b514d596/users/2f5c72447db84ae8818efb70ccd0c1af/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:37.224384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.224793 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.225145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.254792 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] MFA Rules not processed for user `2f5c72447db84ae8818efb70ccd0c1af`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:37.256025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:37.320372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1402/2797] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:37.324765 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.390792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34c1cf-6478-42e4-8e25-c9ff9ac08c0a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.395483 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.396107 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.396394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.396671 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.397870 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.399838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.428751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1396/2798] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:37 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:37.433001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.446388 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78970de-519b-4ccb-ba4c-55af1c00186c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.512196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.512904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.514327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.514327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.514327 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.516574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.550961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1403/2799] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:37 2022] POST /identity/v3/users => generated 307 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:37.554945 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.566818 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc563-f76b-4653-b8a0-230b6d7e2a10 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.572154 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.572840 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.573365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.573744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.574466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.576303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.587473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1397/2800] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:37.591516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.603429 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6f4cf-02e6-4819-838b-cf9ae035afc6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.608424 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.609081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.609365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.609638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] PATH_INFO: `/v3/projects/11382e2a73c2469a828558771bfcab76/users/b7904e7dfe1147238d9afbb0a7043d54/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.625319 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11382e2a73c2469a828558771bfcab76, user_id=b7904e7dfe1147238d9afbb0a7043d54, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.628037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.642222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1404/2801] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:37 2022] PUT /identity/v3/projects/11382e2a73c2469a828558771bfcab76/users/b7904e7dfe1147238d9afbb0a7043d54/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:37.646018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.656388 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d168b9d-bb0f-42f0-ac80-cf11d2165a62 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.727761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.728520 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.728639 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.728910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.729552 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.731181 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca908d-9223-4de3-8c8b-693cb3f83fef admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.745629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1398/2802] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:12:37.747025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.757425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916aae8-aac4-4410-a6a7-62e1ffb0590f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.763461 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.764235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.764624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.765021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] PATH_INFO: `/v3/projects/11382e2a73c2469a828558771bfcab76/users/b7904e7dfe1147238d9afbb0a7043d54/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.767103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11382e2a73c2469a828558771bfcab76, user_id=b7904e7dfe1147238d9afbb0a7043d54, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.770475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.781964 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1405/2803] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:37 2022] PUT /identity/v3/projects/11382e2a73c2469a828558771bfcab76/users/b7904e7dfe1147238d9afbb0a7043d54/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:37.792251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.792543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.792821 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.800016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:37.827809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] MFA Rules not processed for user `b7904e7dfe1147238d9afbb0a7043d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:37.829079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:37.899130 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1399/2804] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:37 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:37.909221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:37.910928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:37.934411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:37.972675 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc8c54-1c3e-48ee-a231-5d393c4e2b34 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:37.976973 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:37.977580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:37.977893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:37.978166 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:37.987395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:37.988827 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412a9d5-ec89-4b48-a850-1cbb7729fe8c service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:37.999836 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1406/2805] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:37 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:38.082849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:38.092856 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7effad-bcb6-4525-814a-26d4073b7440 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:38.096584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:38.098387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:38.098629 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:38.098896 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:38.156037 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:38.157398 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:38.166609 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1400/2806] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:40.300395 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3c2c-0a10-402c-83ac-652eae39b7d5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:40.300885 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3c2c-0a10-402c-83ac-652eae39b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:40.301353 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c3c2c-0a10-402c-83ac-652eae39b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:40.325913 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ed5c3c2c-0a10-402c-83ac-652eae39b7d5 None None] MFA Rules not processed for user `7af84fe0a9534f2da43e238124c430db`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:40.327459 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ed5c3c2c-0a10-402c-83ac-652eae39b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:40.401773 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1407/2807] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:42.222365 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:42.295852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc860dd-0261-4a1a-bd97-517080d39a85 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:42.302812 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:42.303916 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:42.304272 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:42.304633 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] PATH_INFO: `/v3/users/2f5c72447db84ae8818efb70ccd0c1af` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:42.306176 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f5c72447db84ae8818efb70ccd0c1af)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:42.308262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:42.344939 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:42.374802 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f5c72447db84ae8818efb70ccd0c1af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:42.376610 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f5c72447db84ae8818efb70ccd0c1af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:42.376886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f5c72447db84ae8818efb70ccd0c1af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:42.388396 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f5c72447db84ae8818efb70ccd0c1af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:42.413611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f5c72447db84ae8818efb70ccd0c1af'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:42.437556 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1401/2808] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:42 2022] DELETE /identity/v3/users/2f5c72447db84ae8818efb70ccd0c1af => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:42.442110 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:42.442629 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:42.442754 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:42.464604 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:42.466083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:42.541389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1408/2809] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:42.551104 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:42.552695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:42.638624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375b3d4-6c2d-4de6-80bc-1df3cc2cfa77 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:42.643363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:42.644976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:42.645790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:42.646168 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:42.656587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:42.658079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:42.668048 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1402/2810] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:42.968161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:42.970307 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:42.983914 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7020-f0dc-4a10-a7a7-9a35ef606f04 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:42.989096 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:42.989959 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:42.990339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:42.990710 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] PATH_INFO: `/v3/projects/a09a05f6a6ba48c898ea6bd5b514d596` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:42.992939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09a05f6a6ba48c898ea6bd5b514d596)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:42.994992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:43.045484 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09a05f6a6ba48c898ea6bd5b514d596', 'request_id': 'req-169d09dd-0885-48a4-b4fa-c466e12d3db0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.052988 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09a05f6a6ba48c898ea6bd5b514d596', 'request_id': 'req-169d09dd-0885-48a4-b4fa-c466e12d3db0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.060485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09a05f6a6ba48c898ea6bd5b514d596', 'request_id': 'req-169d09dd-0885-48a4-b4fa-c466e12d3db0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.063173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:43.074195 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] The token cache is being invalidate because project a09a05f6a6ba48c898ea6bd5b514d596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:43.074487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.075137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.076627 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1409/2811] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:42 2022] DELETE /identity/v3/projects/a09a05f6a6ba48c898ea6bd5b514d596 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:43.081143 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:43.090911 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec632ad-c629-4795-b4eb-14b01dbedd0c service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:43.152547 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:43.153560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.153886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.154161 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] PATH_INFO: `/v3/users/b7904e7dfe1147238d9afbb0a7043d54` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.155318 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7904e7dfe1147238d9afbb0a7043d54)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:43.157440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:43.210567 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7904e7dfe1147238d9afbb0a7043d54'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.211236 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7904e7dfe1147238d9afbb0a7043d54'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.211910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7904e7dfe1147238d9afbb0a7043d54'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.214602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:43.221986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7904e7dfe1147238d9afbb0a7043d54'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.242289 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d50f91f2-4b45-4743-88af-f0566873fb30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7904e7dfe1147238d9afbb0a7043d54'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.257550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1403/2812] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:12:43 2022] DELETE /identity/v3/users/b7904e7dfe1147238d9afbb0a7043d54 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:43.533686 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:43.535431 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:43.550329 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d09dd-0885-48a4-b4fa-c466e12d3db0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:43.623284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:43.623921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.624202 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.624473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] PATH_INFO: `/v3/projects/11382e2a73c2469a828558771bfcab76` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.625586 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11382e2a73c2469a828558771bfcab76)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:43.627511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:43.682651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11382e2a73c2469a828558771bfcab76', 'request_id': 'req-c63c5e23-8b0f-4f76-a573-113190cbf022'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.690871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11382e2a73c2469a828558771bfcab76', 'request_id': 'req-c63c5e23-8b0f-4f76-a573-113190cbf022'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.698620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11382e2a73c2469a828558771bfcab76', 'request_id': 'req-c63c5e23-8b0f-4f76-a573-113190cbf022'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.711416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] The token cache is being invalidate because project 11382e2a73c2469a828558771bfcab76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:12:43.711416 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.712777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c63c5e23-8b0f-4f76-a573-113190cbf022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:12:43.714068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1410/2813] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:12:43 2022] DELETE /identity/v3/projects/11382e2a73c2469a828558771bfcab76 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:12:43.718668 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.718959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.719293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.738980 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:43.740264 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:43.802163 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1404/2814] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:43.811901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.812256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.812741 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.835725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:43.837111 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:43.848784 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1411/2815] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:43.850997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:43.861549 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa559f4-eb8f-4f28-8721-07cbb397f0e1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:43.866885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:43.867510 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.867935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.868217 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.868876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:12:43.869536 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:43.871727 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:43.887384 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1405/2816] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:12:43.888583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:43.898807 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fd59e-c186-45a3-a678-5da3e7b811a1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:43.904978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:43.905592 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:43.905909 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:43.906185 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:43.906961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:43.908878 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:43.940178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1412/2817] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:12:43 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:43.944465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:43.956789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e5dc3-704d-416c-ab18-37385170f948 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:44.025339 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:44.025987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.026281 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.026545 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.027189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:44.029101 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:44.050640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1406/2818] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:12:43 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:12:44.055364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:44.066495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcffb9-7163-4a93-af24-e82c15a4283c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:44.072139 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:44.072854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.073135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.073405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.074282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:44.076819 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:44.088391 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1413/2819] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:44.092459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:44.103593 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d011ce-9188-4e87-8b85-7ea6f89e7fd3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:44.108448 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:44.109057 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.109746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.110001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.110662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:44.112262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa626a-ae8e-4cc4-90ee-44f0e597e494 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:44.124590 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1407/2820] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:12:44.129618 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:44.147078 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a127b-85cc-432b-a775-ccc09e291447 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:44.152036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:44.152607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.152887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.153161 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] PATH_INFO: `/v3/projects/957dee6ab34040aab9015d2e5718cdde/users/323c1ddd8e7f48b189cba735cf14c795/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.168135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=957dee6ab34040aab9015d2e5718cdde, user_id=323c1ddd8e7f48b189cba735cf14c795, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:44.170802 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:44.183740 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1414/2821] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:12:44 2022] PUT /identity/v3/projects/957dee6ab34040aab9015d2e5718cdde/users/323c1ddd8e7f48b189cba735cf14c795/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:12:44.194115 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295428f-c462-436a-91c0-85f340c3673f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.194411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295428f-c462-436a-91c0-85f340c3673f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.194694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295428f-c462-436a-91c0-85f340c3673f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.222975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f295428f-c462-436a-91c0-85f340c3673f None None] MFA Rules not processed for user `323c1ddd8e7f48b189cba735cf14c795`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:12:44.224216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f295428f-c462-436a-91c0-85f340c3673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:44.244969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:44.299992 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1408/2822] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:12:44.311996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:44.313677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:44.336173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:44.427033 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79106254-207c-4790-891a-18684bbf0eae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:44.431894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:44.432608 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:44.432901 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:44.433540 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:44.442562 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:44.444049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:44.453551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1415/2823] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:12:45.882760 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295428f-c462-436a-91c0-85f340c3673f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:45.883625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f295428f-c462-436a-91c0-85f340c3673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:45.942974 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295428f-c462-436a-91c0-85f340c3673f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:45.952080 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:45.952954 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:45.953244 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:45.953854 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:45.956296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:46.035224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:46.036774 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:46.049279 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1409/2824] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:47.885849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:12:47.887626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:47.973303 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08bbf1-93bb-46cd-acdd-3195c4abffdd service glance] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:12:47.980900 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:12:47.981598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:12:47.981916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:12:47.982345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:12:47.984573 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:12:48.000952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:12:48.002952 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:12:48.014891 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1416/2825] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:12:56.042589 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=67 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:12:58.012086 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=61 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:03.926415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:03.928321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:04.000776 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0bab-07f4-43ac-a800-ddd81930c262 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.005435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.006096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.006382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.006659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] PATH_INFO: `/v3/users/7af84fe0a9534f2da43e238124c430db` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.007806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af84fe0a9534f2da43e238124c430db)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.012112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.067965 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af84fe0a9534f2da43e238124c430db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.068844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af84fe0a9534f2da43e238124c430db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.069492 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af84fe0a9534f2da43e238124c430db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.077018 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af84fe0a9534f2da43e238124c430db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.091851 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-348ef737-52bf-4621-8b4e-31ddda6c8de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af84fe0a9534f2da43e238124c430db'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.106764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1410/2826] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:03 2022] DELETE /identity/v3/users/7af84fe0a9534f2da43e238124c430db => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:04.308207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.322335 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98cf4b-8647-469f-9013-1e2916d2c082 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.390759 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.391389 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.391670 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.391942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] PATH_INFO: `/v3/projects/82ff53810f744f7d96126d1eed8662c5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.393050 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ff53810f744f7d96126d1eed8662c5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.394968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.444407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ff53810f744f7d96126d1eed8662c5', 'request_id': 'req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.453950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ff53810f744f7d96126d1eed8662c5', 'request_id': 'req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.460345 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ff53810f744f7d96126d1eed8662c5', 'request_id': 'req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.473595 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] The token cache is being invalidate because project 82ff53810f744f7d96126d1eed8662c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:04.473922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.474568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7b0b7d94-5e9a-47b5-89b8-24c37cdb327e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:04.476060 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1417/2827] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:04 2022] DELETE /identity/v3/projects/82ff53810f744f7d96126d1eed8662c5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:04.481087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.481373 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.481645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.505993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:04.507229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:04.574991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1411/2828] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:04.583174 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.583472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.583876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.604551 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:04.605816 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:04.613815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1418/2829] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:13:04.618659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.628457 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78ead9-723c-498b-9b72-462e5e88ac01 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.633142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.633775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.634063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.634340 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.634989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:13:04.635297 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.636899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.648712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1412/2830] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:13:04.653020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.663978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa46fd-f3aa-4336-a68a-67b7899a8eed None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.669862 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.670411 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.670701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.670978 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.671607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.673513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.704379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1419/2831] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:04 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:04.708626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.719807 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5be968-239b-41e0-bdfb-50fd4c9a9669 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.723427 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:04.786094 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.786844 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.787141 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.787559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.789210 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.791256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.815985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1413/2832] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:04 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:04.820125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.830745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737cc6d-c009-4b62-922c-a4fce9742586 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.835602 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.836196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.836476 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.836767 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.837400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.839036 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.848732 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1420/2833] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:04.853008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.862869 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50445820-5907-4b72-a899-9410d19e0c89 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.868069 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.868653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.868937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.869211 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.869890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.871512 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74c918-df6d-44d2-8499-0ef72a51b154 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.882432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1414/2834] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:04.886386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:04.896214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3878cb-3ccc-4c90-8007-14e45ce0394f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:04.901258 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:04.901925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.902211 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.902489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] PATH_INFO: `/v3/projects/00c008f3f3e046d696fb50ac1807ac39/users/de930ad62e01422c8073a374c2d7307a/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.917069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c008f3f3e046d696fb50ac1807ac39, user_id=de930ad62e01422c8073a374c2d7307a, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:04.919749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:04.933070 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1421/2835] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:04 2022] PUT /identity/v3/projects/00c008f3f3e046d696fb50ac1807ac39/users/de930ad62e01422c8073a374c2d7307a/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:13:04.943083 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:04.943582 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:04.943683 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:04.971611 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] MFA Rules not processed for user `de930ad62e01422c8073a374c2d7307a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:04.972809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:05.035958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1415/2836] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:05.048318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:05.050049 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:05.080040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:05.110656 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7877e3b-a15b-40dc-8cf1-c749169d18e4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:05.114222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:05.114817 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:05.115094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:05.115366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:05.123568 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:05.124985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135851a3-a714-4055-8b33-9dd6056e303f service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:05.133564 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1422/2837] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:09.788846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:09.865424 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7746-9a82-4384-8d50-85198f48bbaf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:09.870571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:09.871301 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:09.871568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:09.871839 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] PATH_INFO: `/v3/users/de930ad62e01422c8073a374c2d7307a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:09.872976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de930ad62e01422c8073a374c2d7307a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:09.874968 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:09.895180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:09.931442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de930ad62e01422c8073a374c2d7307a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:09.932089 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de930ad62e01422c8073a374c2d7307a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:09.932706 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de930ad62e01422c8073a374c2d7307a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:09.940216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de930ad62e01422c8073a374c2d7307a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:09.953375 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de930ad62e01422c8073a374c2d7307a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:09.966472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1416/2838] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:09 2022] DELETE /identity/v3/users/de930ad62e01422c8073a374c2d7307a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:09.971056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:09.971516 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:09.971616 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:09.990403 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:09.991601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:10.049981 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1423/2839] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:10.060977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.063806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:10.122191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4bc4e-ffd4-4a1d-aec0-897d3ac91432 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.125928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.126533 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.126846 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.127122 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.137298 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.138805 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec77bc5-fa2b-4834-9313-ee3aa4e74b23 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.148892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1417/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:10.416207 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.417633 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:10.429615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28225d-b966-4c82-93eb-78ba2292a976 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.434369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.435001 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.435284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.435556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] PATH_INFO: `/v3/projects/00c008f3f3e046d696fb50ac1807ac39` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.436714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c008f3f3e046d696fb50ac1807ac39)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.438695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.484835 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c008f3f3e046d696fb50ac1807ac39', 'request_id': 'req-291d3d8d-0a2b-4962-bd3d-a64020414b92'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:10.492063 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c008f3f3e046d696fb50ac1807ac39', 'request_id': 'req-291d3d8d-0a2b-4962-bd3d-a64020414b92'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:10.498624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c008f3f3e046d696fb50ac1807ac39', 'request_id': 'req-291d3d8d-0a2b-4962-bd3d-a64020414b92'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:10.511712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] The token cache is being invalidate because project 00c008f3f3e046d696fb50ac1807ac39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:10.512021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:10.512667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-291d3d8d-0a2b-4962-bd3d-a64020414b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:10.514282 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1424/2841] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:10 2022] DELETE /identity/v3/projects/00c008f3f3e046d696fb50ac1807ac39 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:10.519074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.519394 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.519673 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.539419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:10.540626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:10.600206 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1418/2842] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:10.607874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.608172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.608456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.627845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:10.629058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:10.637470 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1425/2843] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:10.641459 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.651597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c5f3-e2d2-482c-a61a-12c2942e581d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.656268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.656859 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.657142 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.657419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.658100 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:13:10.658407 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.660015 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.671548 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1419/2844] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:13:10.675668 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.684774 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85687eb2-65f4-488a-bf2a-2f39b1f2ba8f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.689370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.689990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.690281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.690556 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.691182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.693072 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.720763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1426/2845] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:10 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:10.724745 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.734560 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af483ed-f5f9-42b1-9288-674292578087 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.754317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:10.791601 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.792221 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.792475 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.792747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.793374 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.795345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.816955 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1420/2846] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:10 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:10.821652 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.832145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b4766-4866-49b8-8ef1-b5c26bfa2972 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.837402 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.838023 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.838304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.838647 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.839294 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.840987 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.850972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1427/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:10.854655 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.864399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23353e7-68d2-49d9-a33a-23de4bb3331b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.869507 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.870133 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.870411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.870684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.871317 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.872891 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f2722-34f0-4d14-9cfe-21dee9b8dd3b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.883201 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1421/2848] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:10.887726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:10.897587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070f60-bb04-418c-947d-98bee169eef8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:10.902274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:10.902910 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.903196 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.903474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] PATH_INFO: `/v3/projects/f34d798421304770a1a872e7c0824d35/users/f0c01037a52b4fa586f26183ed30538d/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.915922 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:10.921650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34d798421304770a1a872e7c0824d35, user_id=f0c01037a52b4fa586f26183ed30538d, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:10.924672 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:10.940874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1428/2849] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:10 2022] PUT /identity/v3/projects/f34d798421304770a1a872e7c0824d35/users/f0c01037a52b4fa586f26183ed30538d/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:10.953114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:10.953114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:10.953114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:10.980689 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] MFA Rules not processed for user `f0c01037a52b4fa586f26183ed30538d`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:10.981999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:11.047493 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1422/2850] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:11.060266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:11.062020 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:11.128498 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8ac23-96d7-47e2-89b8-c36de23fc16b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:11.132249 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:11.132859 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:11.133133 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:11.133509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:11.144009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:11.146235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6746-898b-4f7c-97de-01accb546034 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:11.157543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1429/2851] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:20.094096 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:20.095546 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.168135 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7d464-6891-447a-a726-db2e5751a039 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:20.172625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:20.173257 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.173540 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.173855 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] PATH_INFO: `/v3/users/323c1ddd8e7f48b189cba735cf14c795` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.174976 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=323c1ddd8e7f48b189cba735cf14c795)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:20.176900 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:20.215088 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:20.228486 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '323c1ddd8e7f48b189cba735cf14c795'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.229166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '323c1ddd8e7f48b189cba735cf14c795'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.229975 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '323c1ddd8e7f48b189cba735cf14c795'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.237454 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '323c1ddd8e7f48b189cba735cf14c795'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.249989 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '323c1ddd8e7f48b189cba735cf14c795'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.263111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1423/2852] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:20 2022] DELETE /identity/v3/users/323c1ddd8e7f48b189cba735cf14c795 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:20.267679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.268182 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.268292 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.288641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:20.289929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.349837 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1430/2853] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:20.359564 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:20.360993 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.424001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e10b9-3a79-416b-b8ac-b4206188ef65 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:20.427617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:20.428225 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.428508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.428785 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.437849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:20.439295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a12f4-34a7-4def-b258-a7c2b8927a74 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:20.449658 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1424/2854] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:20.718092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:20.719227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.737176 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0e2b9-7791-4426-a58e-8e958a8ed510 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:20.746188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:20.746188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.746188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.746188 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] PATH_INFO: `/v3/projects/957dee6ab34040aab9015d2e5718cdde` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.747255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957dee6ab34040aab9015d2e5718cdde)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:20.749330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:20.797455 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957dee6ab34040aab9015d2e5718cdde', 'request_id': 'req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.805508 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957dee6ab34040aab9015d2e5718cdde', 'request_id': 'req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.812369 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957dee6ab34040aab9015d2e5718cdde', 'request_id': 'req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.823928 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] The token cache is being invalidate because project 957dee6ab34040aab9015d2e5718cdde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:20.824225 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.824886 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a2c7fcd-ee32-4519-91e5-25ada62d05d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:20.826468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1431/2855] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:20 2022] DELETE /identity/v3/projects/957dee6ab34040aab9015d2e5718cdde => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:20.831559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.831852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.832158 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.851666 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:20.852887 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.914702 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1425/2856] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:20.923538 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.923821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.924093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.943826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:20.945025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:20.953903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1432/2857] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:20.958137 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:20.967626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccfd61-dbbe-4aaf-9b01-edc2249d251b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:20.972175 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:20.972788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:20.973066 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:20.973337 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:20.974054 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:13:20.974354 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:20.975982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:20.990216 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1426/2858] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:13:20.995099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.004364 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efe610-5ba8-4ea5-9273-37e51c7ca049 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.009006 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.009624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.009949 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.010221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.010845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.012742 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.018121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:21.041870 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1433/2859] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:20 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:21.045985 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.056937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5e102-70ce-46a9-9d06-cc52c2a8821a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.099270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:21.118417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.119028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.119341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.119617 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.120266 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.122219 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.144941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1427/2860] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:21 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:21.149064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.159467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958103e7-c061-4830-969f-9087cab18747 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.164099 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.165220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.165511 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.165821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.166463 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.168061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.177820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1434/2861] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:13:21.182349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.192382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edf856-c430-4c15-894a-2f719c204234 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.197472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.198191 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.198484 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.198781 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.199482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.201178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2237b-3013-42dc-a8be-f72bd0c18757 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.213482 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1428/2862] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:21.217531 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.229289 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ae1a-908c-43fc-b4a9-8f27ec951a61 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.233865 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.234501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.234785 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.235061 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] PATH_INFO: `/v3/projects/09429da1965040399ba3d025e69ba9a2/users/8647b50ebdd2470a8160be476cb79e49/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.250317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09429da1965040399ba3d025e69ba9a2, user_id=8647b50ebdd2470a8160be476cb79e49, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.252963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.266191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1435/2863] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:21 2022] PUT /identity/v3/projects/09429da1965040399ba3d025e69ba9a2/users/8647b50ebdd2470a8160be476cb79e49/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:21.277112 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.277423 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.277736 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.305570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] MFA Rules not processed for user `8647b50ebdd2470a8160be476cb79e49`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:21.306815 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:21.377028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1429/2864] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:21.389000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.390796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:21.449907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489152a-9b39-4608-9d04-f0d49a93cc5c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.453775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.454379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.454657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.454929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.463332 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.464925 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea907267-388e-4c6e-9351-49cf84f31dba service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.473495 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1436/2865] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:21.503324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.569932 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600156c9-8104-42a8-aa40-343a1b387322 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.575157 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.575795 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.576081 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.576361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] PATH_INFO: `/v3/users/8647b50ebdd2470a8160be476cb79e49` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.577777 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8647b50ebdd2470a8160be476cb79e49)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.579718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.626653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8647b50ebdd2470a8160be476cb79e49'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:21.627522 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8647b50ebdd2470a8160be476cb79e49'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:21.628248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8647b50ebdd2470a8160be476cb79e49'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:21.633937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:21.637147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8647b50ebdd2470a8160be476cb79e49'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:21.650382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8647b50ebdd2470a8160be476cb79e49'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:21.662991 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1430/2866] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:21 2022] DELETE /identity/v3/users/8647b50ebdd2470a8160be476cb79e49 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:21.667487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e6018-e639-46f9-a416-a5637153357c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.667782 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e6018-e639-46f9-a416-a5637153357c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.668064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e6018-e639-46f9-a416-a5637153357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.687771 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-630e6018-e639-46f9-a416-a5637153357c None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:21.689128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-630e6018-e639-46f9-a416-a5637153357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:21.759714 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1437/2867] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:21.769686 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:21.771147 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:21.829031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec8af-dc80-4fc5-89b8-0c5f590f1171 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:21.832786 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:21.833408 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:21.833690 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:21.833995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:21.843575 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:21.845009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4651-bf68-4c86-9e61-0f3ec313c9c5 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:21.854952 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1431/2868] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:22.133887 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e6018-e639-46f9-a416-a5637153357c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.135458 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-630e6018-e639-46f9-a416-a5637153357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:22.153695 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e6018-e639-46f9-a416-a5637153357c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.159003 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.159655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.159942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.160216 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] PATH_INFO: `/v3/projects/09429da1965040399ba3d025e69ba9a2` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.161359 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09429da1965040399ba3d025e69ba9a2)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.166365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.218808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09429da1965040399ba3d025e69ba9a2', 'request_id': 'req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:22.220417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:22.227945 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09429da1965040399ba3d025e69ba9a2', 'request_id': 'req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:22.236406 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09429da1965040399ba3d025e69ba9a2', 'request_id': 'req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:22.248630 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] The token cache is being invalidate because project 09429da1965040399ba3d025e69ba9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:22.248920 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:22.249566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c8455aa7-73ac-4c6e-bab5-f7c058334ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:22.251113 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1438/2869] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:22 2022] DELETE /identity/v3/projects/09429da1965040399ba3d025e69ba9a2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:22.256038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.256508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.256596 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.276729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:22.278009 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:22.340682 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1432/2870] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:22.348191 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.348479 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.348755 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.374024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:22.375311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:22.383475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1439/2871] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:22.387762 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.397440 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd67ccd-91af-45d0-bbb2-4f5b905f9b5d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.402645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.403256 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.403524 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.403801 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.404447 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:13:22.404751 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.406395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.418659 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1433/2872] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:13:22.422822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.432679 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca451a-8de5-482f-b4dc-5fecf3e252aa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.437316 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.437946 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.438232 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.438507 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.439143 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.441032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.468813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1440/2873] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:22 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:22.472942 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.483465 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3448d0-6746-4d08-a3ef-a4dff1990788 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.541563 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.542199 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.542488 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.542766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.543400 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.545321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.553190 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:22.573886 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1434/2874] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:22 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:22.578145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.589378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce183d8-b72b-4793-b25e-0979e3f85edf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.594042 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.594583 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.594869 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.595159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.595781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.597372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.607224 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1441/2875] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:22.611182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.620966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365149-5bec-489d-86bd-9d538844b148 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.625558 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.626182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.626461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.626733 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.627366 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.628946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1f718-9400-4115-bdf9-8a94cfef0b9b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.641005 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1435/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:22.645146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.655961 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366820f-1808-42dd-af8b-2a6f2a09448d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.660641 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.661347 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.661632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.661947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] PATH_INFO: `/v3/projects/8d9439873b26425cb1017f4a96a0bd34/users/d6702ce4c1544a85b84547eb4e659eb7/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.676306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9439873b26425cb1017f4a96a0bd34, user_id=d6702ce4c1544a85b84547eb4e659eb7, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.678996 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.691709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1442/2877] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:22 2022] PUT /identity/v3/projects/8d9439873b26425cb1017f4a96a0bd34/users/d6702ce4c1544a85b84547eb4e659eb7/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:22.703404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb87f7-5685-4225-9ca7-33383243632f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.703694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb87f7-5685-4225-9ca7-33383243632f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.703994 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb87f7-5685-4225-9ca7-33383243632f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.736226 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9fdb87f7-5685-4225-9ca7-33383243632f None None] MFA Rules not processed for user `d6702ce4c1544a85b84547eb4e659eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:22.737461 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9fdb87f7-5685-4225-9ca7-33383243632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:22.803467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1436/2878] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:22.814671 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:22.816372 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:22.873976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9734e43-3a38-41db-b6c8-f8bff122ed13 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:22.877587 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44a33c23a10f4ae5a008e1d4d4973355', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:22.878239 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:22.878526 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:22.878805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:22.888440 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:22.889923 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:22.898626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1443/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:24.258860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bbfd4-fccc-4105-b966-a75c25586d5c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:24.259315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bbfd4-fccc-4105-b966-a75c25586d5c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:24.259404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bbfd4-fccc-4105-b966-a75c25586d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:24.286766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-049bbfd4-fccc-4105-b966-a75c25586d5c None None] MFA Rules not processed for user `d6702ce4c1544a85b84547eb4e659eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:24.288008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-049bbfd4-fccc-4105-b966-a75c25586d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:24.295602 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1437/2880] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:32.894599 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:34.280290 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=3,keystone:SELECT=42 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:40.098394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:40.099830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:40.163955 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0218a36-0f18-48c9-bdc5-629b7b4c540a service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:40.170068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:40.171085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:40.171515 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:40.171794 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] PATH_INFO: `/v3/users/f0c01037a52b4fa586f26183ed30538d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:40.172939 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c01037a52b4fa586f26183ed30538d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:40.174897 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:40.225418 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c01037a52b4fa586f26183ed30538d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.226145 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c01037a52b4fa586f26183ed30538d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.226752 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c01037a52b4fa586f26183ed30538d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.233727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c01037a52b4fa586f26183ed30538d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.246439 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c01037a52b4fa586f26183ed30538d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.259447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1444/2881] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:40 2022] DELETE /identity/v3/users/f0c01037a52b4fa586f26183ed30538d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:40.264102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:40.264569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:40.264684 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:40.284240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:40.285049 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:40.351912 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1438/2882] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:40.365469 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:40.367637 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:40.469976 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48877a-f29c-43b8-b061-23c66d00fb8c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:40.474144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:40.474848 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:40.475085 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:40.475360 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:40.486212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:40.487650 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad726a-57c6-487d-b776-2092f8f3dcf1 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:40.498012 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1445/2883] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:13:40.776806 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:40.777941 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:40.790836 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08229b3b-9ac3-4385-9c0c-30651a346e9e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:40.795693 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:40.796368 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:40.796790 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:40.797077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] PATH_INFO: `/v3/projects/f34d798421304770a1a872e7c0824d35` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:40.798316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34d798421304770a1a872e7c0824d35)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:40.800182 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:40.848114 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34d798421304770a1a872e7c0824d35', 'request_id': 'req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.854849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34d798421304770a1a872e7c0824d35', 'request_id': 'req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.860585 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34d798421304770a1a872e7c0824d35', 'request_id': 'req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.872678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] The token cache is being invalidate because project f34d798421304770a1a872e7c0824d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:40.872973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.873587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:40.875172 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1439/2884] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:40 2022] DELETE /identity/v3/projects/f34d798421304770a1a872e7c0824d35 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:41.570763 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1f2fd-7973-4eb2-9ae7-0546e5ba6afc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:41.571728 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1f2fd-7973-4eb2-9ae7-0546e5ba6afc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:41.571838 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1f2fd-7973-4eb2-9ae7-0546e5ba6afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:41.589394 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:41.634172 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-73c1f2fd-7973-4eb2-9ae7-0546e5ba6afc None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:41.635334 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-73c1f2fd-7973-4eb2-9ae7-0546e5ba6afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:41.698642 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1446/2885] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:41.752039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:41.753559 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:41.844951 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e4709-dd18-4026-8fdb-c9946f8ad1f8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:41.850098 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:41.850710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:41.851003 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:41.851285 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:41.856243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:41.860827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:41.863431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:41.872527 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1440/2886] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:13:42.395590 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c4fe-cfd9-4117-94d3-e785ec853af4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:42.395888 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c4fe-cfd9-4117-94d3-e785ec853af4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:42.396180 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c4fe-cfd9-4117-94d3-e785ec853af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:42.415103 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2868c4fe-cfd9-4117-94d3-e785ec853af4 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:42.416238 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2868c4fe-cfd9-4117-94d3-e785ec853af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:42.423324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1447/2887] 10.222.0.20 () {58 vars in 967 bytes} [Mon Sep 12 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:42.435692 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:42.446804 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803712c-9175-490e-933c-875f5095cab3 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:42.451842 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:42.452437 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:42.452973 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:42.453251 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:42.461229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:42.462838 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:42.470810 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1441/2888] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:13:42.697392 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce6d9-062b-483e-b713-0068a33888ac None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:42.697882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce6d9-062b-483e-b713-0068a33888ac None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:42.697966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce6d9-062b-483e-b713-0068a33888ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:42.717100 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-971ce6d9-062b-483e-b713-0068a33888ac None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:42.718377 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-971ce6d9-062b-483e-b713-0068a33888ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:42.725387 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1448/2889] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:42.737726 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:42.748979 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edebd8a-5203-4252-8e4d-4e9612eee108 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:42.754149 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:42.754746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:42.755034 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:42.755315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:42.763729 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:42.765315 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:42.773334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1442/2890] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:13:43.086021 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:43.086311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:43.087081 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:43.105876 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:43.107073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:43.114138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1449/2891] 10.222.0.20 () {58 vars in 967 bytes} [Mon Sep 12 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:43.126472 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:43.137570 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dcadd-9371-49ec-9865-6ae8eec49b6b service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:43.142835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:43.143444 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:43.143730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:43.144011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:43.152160 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:43.153775 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6869-9ee1-4e00-a1d7-3a272c75f9db service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:43.161761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1443/2892] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:13:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 16:13:45.227850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:45.229285 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.293486 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c3993-3209-41b2-b257-ff29b6481861 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:45.298054 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:45.298682 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.298965 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.299241 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] PATH_INFO: `/v3/users/d6702ce4c1544a85b84547eb4e659eb7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.300361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6702ce4c1544a85b84547eb4e659eb7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:45.302314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:45.347918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6702ce4c1544a85b84547eb4e659eb7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.348559 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6702ce4c1544a85b84547eb4e659eb7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.349164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6702ce4c1544a85b84547eb4e659eb7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.355960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6702ce4c1544a85b84547eb4e659eb7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.357080 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:45.370093 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6702ce4c1544a85b84547eb4e659eb7'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.383041 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1450/2893] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:13:45 2022] DELETE /identity/v3/users/d6702ce4c1544a85b84547eb4e659eb7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:45.388511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.388977 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.389079 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.407718 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:45.408922 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.468071 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1444/2894] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:45.477661 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:45.479082 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.534962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ce19-6755-4248-967e-55a025b4880a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:45.538561 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:45.539156 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.539437 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.539709 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.548448 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:45.549899 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17461c4e-8856-45aa-bff4-c61c2cad7599 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:45.559521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1451/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:45.729814 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:45.731360 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.744419 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e6a3-65c5-4afc-b7d0-2f79011c5cb4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:45.751877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:45.753621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.753621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.753621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] PATH_INFO: `/v3/projects/8d9439873b26425cb1017f4a96a0bd34` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.754043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9439873b26425cb1017f4a96a0bd34)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:45.755959 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:45.803950 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9439873b26425cb1017f4a96a0bd34', 'request_id': 'req-b79a30e0-0e7e-43d1-9320-c20de89babc9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.810879 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9439873b26425cb1017f4a96a0bd34', 'request_id': 'req-b79a30e0-0e7e-43d1-9320-c20de89babc9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.816899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9439873b26425cb1017f4a96a0bd34', 'request_id': 'req-b79a30e0-0e7e-43d1-9320-c20de89babc9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.829640 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] The token cache is being invalidate because project 8d9439873b26425cb1017f4a96a0bd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:13:45.829958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.830571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b79a30e0-0e7e-43d1-9320-c20de89babc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:13:45.832146 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1445/2896] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:13:45 2022] DELETE /identity/v3/projects/8d9439873b26425cb1017f4a96a0bd34 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:45.837164 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.837550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.837953 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.859101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:45.860302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.920918 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1452/2897] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:45.929927 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.930764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.931046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.950730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:45.951921 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:45.959895 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1446/2898] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:45.964011 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:45.973749 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeeb1b-b228-41e6-89a4-e3a29498582e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:45.978705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:45.979295 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:45.979579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:45.979854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:45.980525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:13:45.980826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:45.982467 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:45.994598 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1453/2899] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 16:13:45.998834 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.008144 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5c16-5483-4f8b-b636-6bdac03bd8b4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.012757 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.013349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.013628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.013933 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.014571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.016466 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.044652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1447/2900] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:45 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:46.048726 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.059341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46179cb-09a3-4be7-ad7e-481f35ab4ba6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.114760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.115352 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.115632 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.115903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.116539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.118510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.128101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:46.140898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1454/2901] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:46 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:46.144966 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.155704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ccca-3803-4330-bfb5-ef78e8de82c3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.160526 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.161121 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.161402 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.161711 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.162415 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.164031 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.166694 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:46.176076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1448/2902] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:46.180181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.190419 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd76b58-726b-49b7-baa5-a7fd3384dcad admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.195170 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.195760 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.196040 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.196314 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.196948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.198566 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a60d5-a6dd-47b4-8bed-1a1817a0d8ff admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.209209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1455/2903] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:46.213937 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.223648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9767c-9a90-4629-b66e-ac44642cbb3b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.228784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.229363 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.229641 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.229949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] PATH_INFO: `/v3/projects/41929b1bcb13430cbcf928cfb6b2c939/users/61e2422509e8472795b32b77cea200cb/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.245117 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41929b1bcb13430cbcf928cfb6b2c939, user_id=61e2422509e8472795b32b77cea200cb, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.247787 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.260595 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1449/2904] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:46 2022] PUT /identity/v3/projects/41929b1bcb13430cbcf928cfb6b2c939/users/61e2422509e8472795b32b77cea200cb/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:46.265462 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.265930 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.266119 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.285228 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:46.286496 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:46.345727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1456/2905] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:46.355987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.357725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:46.413608 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576063-993a-47ed-a4d7-8be622cfc051 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:46.417910 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:46.418513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:46.418796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:46.419070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:46.428295 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:46.429755 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc36aa8-365f-44ab-930e-8a07446a8cf8 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:46.439017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1450/2906] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:46.981880 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:46.984168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:46.996674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742c486-cd94-4ffb-ab41-5065cd65b920 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:47.000528 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:47.001317 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:47.001601 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:47.001974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:47.003696 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:47.014324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:47.015788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:47.024745 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1457/2907] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:50.528012 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.528517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.528803 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.566885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] MFA Rules not processed for user `61e2422509e8472795b32b77cea200cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:50.568399 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:50.641433 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1451/2908] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:50.646154 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.658667 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b98e5-055c-402c-912e-e86c5e360c50 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.663212 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.663822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.664107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.664384 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.665068 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:50.667025 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:50.696845 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1458/2909] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:13:50 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:13:50.701017 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.712473 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b4226-53dd-4060-b452-6279ccfa5233 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.775428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.776038 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.776321 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.776600 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.777243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:50.779224 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:50.790949 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:50.802268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1452/2910] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:13:50 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:13:50.806830 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.817121 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507789-bbb0-4eef-a31e-4bb0e90de404 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.822503 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.823104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.823386 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.823660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.824302 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:50.825942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:50.836442 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1459/2911] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:50.840568 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.850680 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f034-e5da-4985-ba73-28e86bc1ceeb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.855405 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.856095 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.856367 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.856643 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] PATH_INFO: `/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc/users/3fa6901b7cc846ec800260453048932c/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.872004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8cfa5315194a3cbfdda93c98d2e7dc, user_id=3fa6901b7cc846ec800260453048932c, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:50.874737 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:50.888345 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1453/2912] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:50 2022] PUT /identity/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc/users/3fa6901b7cc846ec800260453048932c/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:50.892321 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.902474 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeff2b4-a8e0-4f10-b382-1c2a2546b681 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.960992 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.961631 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.961989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.962268 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.962940 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:50.964521 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2c9e2-2a8f-409a-aabb-8073ab3822e0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:50.974179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1460/2913] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:13:50.978938 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:50.989303 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892c69-dc68-4eeb-96d0-be8069e9677a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:50.994744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:50.995432 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:50.995724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:50.996000 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] PATH_INFO: `/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc/users/3fa6901b7cc846ec800260453048932c/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:50.997967 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8cfa5315194a3cbfdda93c98d2e7dc, user_id=3fa6901b7cc846ec800260453048932c, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:51.000605 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:51.010852 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1454/2914] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:13:50 2022] PUT /identity/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc/users/3fa6901b7cc846ec800260453048932c/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:13:55.582325 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c211a0c-e207-46d1-b51c-ea25213156f0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:55.582805 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c211a0c-e207-46d1-b51c-ea25213156f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:55.582895 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c211a0c-e207-46d1-b51c-ea25213156f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:55.614475 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0c211a0c-e207-46d1-b51c-ea25213156f0 None None] MFA Rules not processed for user `3fa6901b7cc846ec800260453048932c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:55.615736 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0c211a0c-e207-46d1-b51c-ea25213156f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:55.637447 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:13:55.700146 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1461/2915] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:55.713739 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:55.715625 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:55.780008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2782a7-04bf-46d9-a343-11edcf232304 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:55.783730 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ae494b13b5743a2b56054d2fae338fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:55.784348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:55.784626 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:55.784897 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:55.845899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:55.847361 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:55.859166 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1455/2916] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:13:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:13:56.011777 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d526a-50f3-4f6d-9426-f0a91bd30af0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:56.012168 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d526a-50f3-4f6d-9426-f0a91bd30af0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:56.012453 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d526a-50f3-4f6d-9426-f0a91bd30af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:56.034700 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a42d526a-50f3-4f6d-9426-f0a91bd30af0 None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:13:56.036101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a42d526a-50f3-4f6d-9426-f0a91bd30af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:56.100971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1462/2917] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:13:56.114028 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:13:56.115571 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:13:56.204649 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566fd2-03bc-426f-89ec-6495c9e887b3 service glance] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:13:56.209797 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:13:56.210414 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:13:56.210697 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:13:56.210971 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:13:56.219621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:13:56.221227 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeea60-e42a-4fb0-857b-05db90f01b07 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:13:56.230331 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1456/2918] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:13:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:14:04.804585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:14:04.805184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:14:04.805184 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:14:04.825786 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] MFA Rules not processed for user `61e2422509e8472795b32b77cea200cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:14:04.826994 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:14:04.834288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1463/2919] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:14:06.226232 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:14:06.882782 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:14:06.883046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:14:06.883327 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:14:06.902764 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:14:06.904072 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:14:06.911404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1457/2920] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Sep 12 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:14:06.924966 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:14:06.938039 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cda64-d71d-443c-9adf-19e8aec009db None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:14:06.943076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:14:06.943674 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:14:06.943956 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:14:06.944227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:14:06.952460 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:14:06.954365 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:14:06.964135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1464/2921] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:14:16.897111 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:14:16.959712 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=53 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:14:32.458758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:14:32.514262 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725dc1e-cf4f-454b-b29f-0c42ad7d9bcf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:14:32.517837 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:14:32.518442 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:14:32.518724 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:14:32.518999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:14:32.529039 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:14:32.531087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:14:32.541664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1458/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:14:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:14:42.536808 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=26 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:14:56.329963 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:14:56.339919 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c834e6-4032-4dfd-a78b-9b3da0dd1fae service swift] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:14:56.343488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:14:56.344076 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:14:56.344361 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:14:56.344638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:14:56.346357 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:14:56.417342 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:14:56.418813 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:14:56.430031 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1465/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:14:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:15:06.424796 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=34 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:15:10.011645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:15:10.020387 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6ebf-a58c-404f-9c93-ab4d0aab966e service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:15:10.024013 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:15:10.024621 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:15:10.024903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:15:10.025342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:15:10.027233 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:15:10.037441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:15:10.039067 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:15:10.048669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1459/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:15:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:15:10.133037 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:15:10.141980 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940e65-8c07-4d9b-bd76-deb34b5e08d2 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:15:10.145694 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:15:10.146404 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:15:10.146762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:15:10.147044 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:15:10.149274 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:15:10.162657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:15:10.164019 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4169c-ce57-48d9-9c17-6b339609e003 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:15:10.175761 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1466/2925] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:15:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:15:20.044574 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:15:20.170790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:15:42.242324 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:15:42.243738 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:15:42.339822 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64919de-2ad0-4156-aaae-c8ca7029f146 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:15:42.344872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:15:42.345476 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:15:42.345788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:15:42.346063 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:15:42.347789 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:15:42.359178 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:15:42.360788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:15:42.371704 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1460/2926] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:15:52.366678 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=41 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:15:57.002702 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:15:57.003149 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:15:57.003243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:15:57.023488 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] MFA Rules not processed for user `b232365f81bc4933bec7a8a86778a494`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:15:57.024683 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:15:57.031765 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1467/2927] 10.222.0.20 () {58 vars in 967 bytes} [Mon Sep 12 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:15:57.074304 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:15:57.086662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9843de-a000-4d4a-ada4-07d3e4e201a0 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:15:57.091747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9679f016e524268b81488141bfc8777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:15:57.092341 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:15:57.092618 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:15:57.092889 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:15:57.100946 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:15:57.102554 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:15:57.110528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1461/2928] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Sep 12 16:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 16:16:03.379318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:03.394689 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff0e34-638b-4cf4-bedd-0b2fcad82fcc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:03.410064 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:03.410874 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:03.411306 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:03.411748 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:03.413815 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:03.428472 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:03.434339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:03.439630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:03.447251 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1468/2929] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:16:03.530078 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f414e-279a-4679-9d07-07eef1246a32 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:03.535302 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:03.535995 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:03.536276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:03.536553 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] PATH_INFO: `/v3/users/61e2422509e8472795b32b77cea200cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:03.537688 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e2422509e8472795b32b77cea200cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:03.539721 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:03.590562 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e2422509e8472795b32b77cea200cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.591208 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e2422509e8472795b32b77cea200cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.591827 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e2422509e8472795b32b77cea200cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.599209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e2422509e8472795b32b77cea200cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.611248 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e2422509e8472795b32b77cea200cb'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.623441 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1462/2930] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:16:03 2022] DELETE /identity/v3/users/61e2422509e8472795b32b77cea200cb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:16:03.800638 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:03.811137 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6917f6-c83b-40b8-a533-7b8eb14e8c3f service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:03.868231 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:03.868854 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:03.869136 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:03.869410 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] PATH_INFO: `/v3/projects/41929b1bcb13430cbcf928cfb6b2c939` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:03.870550 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=41929b1bcb13430cbcf928cfb6b2c939)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:03.872438 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:03.921162 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41929b1bcb13430cbcf928cfb6b2c939', 'request_id': 'req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.928849 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41929b1bcb13430cbcf928cfb6b2c939', 'request_id': 'req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.935017 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41929b1bcb13430cbcf928cfb6b2c939', 'request_id': 'req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.957032 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] The token cache is being invalidate because project 41929b1bcb13430cbcf928cfb6b2c939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:16:03.957323 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.957972 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:03.959660 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1469/2931] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:16:03 2022] DELETE /identity/v3/projects/41929b1bcb13430cbcf928cfb6b2c939 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:03.963404 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:03.964860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:03.978435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf05d9-da7d-4187-ad34-d859889422a4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:04.027348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:04.041056 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:04.041725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:04.042043 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:04.042319 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] PATH_INFO: `/v3/users/3fa6901b7cc846ec800260453048932c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:04.043490 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa6901b7cc846ec800260453048932c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:04.045676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:04.094212 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa6901b7cc846ec800260453048932c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.095376 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa6901b7cc846ec800260453048932c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.095987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa6901b7cc846ec800260453048932c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.103204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa6901b7cc846ec800260453048932c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.115758 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa6901b7cc846ec800260453048932c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.130029 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1463/2932] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:16:03 2022] DELETE /identity/v3/users/3fa6901b7cc846ec800260453048932c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:04.297304 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:04.298882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:04.312014 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d576-c1e0-41d5-9666-5d309f6cd8fe admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:04.368706 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:04.369348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:04.369651 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:04.369960 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] PATH_INFO: `/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:04.371114 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8cfa5315194a3cbfdda93c98d2e7dc)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:04.373214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:04.374738 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:04.422171 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8cfa5315194a3cbfdda93c98d2e7dc', 'request_id': 'req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.429341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8cfa5315194a3cbfdda93c98d2e7dc', 'request_id': 'req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.435974 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8cfa5315194a3cbfdda93c98d2e7dc', 'request_id': 'req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.448489 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] The token cache is being invalidate because project 2c8cfa5315194a3cbfdda93c98d2e7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:16:04.448769 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.449417 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:04.450936 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1470/2933] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:16:04 2022] DELETE /identity/v3/projects/2c8cfa5315194a3cbfdda93c98d2e7dc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:14.126731 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:14.445350 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:40.824768 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:40.827306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:40.918196 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2562fb1-b24e-4728-8afe-f156b7ecff4e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:40.994234 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:40.994981 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:40.995865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:40.995865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:40.997565 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:41.003585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:41.004968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:41.011189 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:41.012796 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:41.025664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1464/2934] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:16:41.074527 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb6d2-c8b9-4b6c-a9d8-9f36320cb155 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:41.078209 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:41.079355 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:41.079924 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:41.080204 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:41.082115 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:41.098398 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:41.101466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:41.114339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1471/2935] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:16:45.090636 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:45.100987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f7a8f-d5f5-4c6c-ae28-44c1c9c37a55 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:45.104628 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:45.105228 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:45.105514 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:45.105833 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:45.107566 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:45.173913 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:45.175273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:45.185153 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1465/2936] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:16:51.108871 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=26 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:51.116894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:51.118240 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:51.169328 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:51.170885 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:51.189070 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c07d3-dd48-4e08-be4b-ad7d85a4e45a service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:51.194108 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:51.195077 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:51.195377 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:51.195661 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] PATH_INFO: `/v3/users/deb44b514dbf425aa108690d647179f4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:51.196865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb44b514dbf425aa108690d647179f4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:51.198899 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:51.202375 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84a09a-4f7d-4406-a438-9e8711dfe535 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:51.208107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:51.208820 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:51.209101 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:51.209509 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:51.211512 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:51.214798 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:51.225543 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:51.227257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:51.240655 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1472/2937] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:16:51.268330 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb44b514dbf425aa108690d647179f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.269033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb44b514dbf425aa108690d647179f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.269656 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb44b514dbf425aa108690d647179f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.277864 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb44b514dbf425aa108690d647179f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.291021 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb44b514dbf425aa108690d647179f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.304935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1466/2938] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:16:51 2022] DELETE /identity/v3/users/deb44b514dbf425aa108690d647179f4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 16:16:51.478311 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:51.489186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ddf06-2cde-4005-9545-6606727525ee service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:51.546947 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:51.547580 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:51.547866 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:51.548144 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] PATH_INFO: `/v3/projects/de833de2ba294244859ccbe233ced7ad` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:51.549257 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de833de2ba294244859ccbe233ced7ad)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:51.551256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:51.597466 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de833de2ba294244859ccbe233ced7ad', 'request_id': 'req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.605916 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de833de2ba294244859ccbe233ced7ad', 'request_id': 'req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.612091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de833de2ba294244859ccbe233ced7ad', 'request_id': 'req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.622056 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] The token cache is being invalidate because project de833de2ba294244859ccbe233ced7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:16:51.622341 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.622967 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.624482 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1473/2939] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:16:51 2022] DELETE /identity/v3/projects/de833de2ba294244859ccbe233ced7ad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:51.628496 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:51.629983 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:51.643550 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62f1b-c030-4e14-9d4e-46dd7dc192c0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:51.699240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:51.699809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:51.700093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:51.700369 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] PATH_INFO: `/v3/users/42ce3147f4234c3ebc275bf697e332b6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:51.701498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ce3147f4234c3ebc275bf697e332b6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:51.703987 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:51.750652 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ce3147f4234c3ebc275bf697e332b6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.751293 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ce3147f4234c3ebc275bf697e332b6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.751893 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ce3147f4234c3ebc275bf697e332b6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.758835 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ce3147f4234c3ebc275bf697e332b6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.771033 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-63bf7213-074d-46d2-a87e-c7b190b8cf7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ce3147f4234c3ebc275bf697e332b6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:51.783799 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1467/2940] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:16:51 2022] DELETE /identity/v3/users/42ce3147f4234c3ebc275bf697e332b6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:51.951281 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:51.954138 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:51.977775 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535eef-1bcb-448a-8c85-e9ff5c1ba13b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:51.995288 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:52.038309 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.038941 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.039221 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.039494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] PATH_INFO: `/v3/projects/090a9576cf6a4341986e32e24592e05c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.040589 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090a9576cf6a4341986e32e24592e05c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.042565 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.092255 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090a9576cf6a4341986e32e24592e05c', 'request_id': 'req-4009a639-a917-4a3e-a1dd-fd33657739c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:52.099477 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090a9576cf6a4341986e32e24592e05c', 'request_id': 'req-4009a639-a917-4a3e-a1dd-fd33657739c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:52.106000 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090a9576cf6a4341986e32e24592e05c', 'request_id': 'req-4009a639-a917-4a3e-a1dd-fd33657739c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:52.116108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] The token cache is being invalidate because project 090a9576cf6a4341986e32e24592e05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:16:52.116677 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:52.117035 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4009a639-a917-4a3e-a1dd-fd33657739c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:16:52.118533 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1474/2941] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:16:51 2022] DELETE /identity/v3/projects/090a9576cf6a4341986e32e24592e05c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:52.123662 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.123867 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.124255 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.137073 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:52.194349 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:16:52.195584 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:52.256877 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1468/2942] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:16:52.265788 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.266104 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.266379 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.286071 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:16:52.287283 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:52.295235 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1475/2943] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:16:52.300008 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.309597 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c27bd6-6b56-4494-bfb1-808ce5fe206f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.314788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.315386 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.315669 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.315944 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.316586 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 16:16:52.316892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.318531 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.330229 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1469/2944] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Sep 12 16:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 16:16:52.334246 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.343186 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb33fd-8d71-4661-8d90-48be8a886702 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.347821 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.348405 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.348688 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.348968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.349596 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.351691 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.389339 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1476/2945] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:16:52 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:16:52.393691 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.403543 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bafd-a947-4501-8f6d-0687566646db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.458710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.459240 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.459519 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.459788 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.460411 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.462346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.482928 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1470/2946] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:16:52 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:16:52.487522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.497456 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b701b5-a0d2-48ab-8b81-e2e5687e0f32 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.502574 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.503159 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.503435 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.503705 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.504336 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.505950 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.516485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1477/2947] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:16:52.520306 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.530030 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25c12-b68d-4745-b4bb-8c388fff8785 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.534588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.535171 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.535451 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.535723 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.536353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.537963 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb0a40-1e75-4d61-a3b8-815d745998de admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.548766 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1471/2948] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:16:52.552539 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.562338 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e29d-3cdd-4e2e-ae0a-86a0aeab19ce admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.567441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.568073 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.568351 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.568624 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] PATH_INFO: `/v3/projects/b0664ad8cda546e59bcadd62db443bfc/users/1f681821823b4a2a8a9a30a41123b36a/roles/86d42fd805a14d5d8e83fbb0653bdb3e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.583287 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0664ad8cda546e59bcadd62db443bfc, user_id=1f681821823b4a2a8a9a30a41123b36a, role_id=86d42fd805a14d5d8e83fbb0653bdb3e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.585948 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.599685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1478/2949] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:16:52 2022] PUT /identity/v3/projects/b0664ad8cda546e59bcadd62db443bfc/users/1f681821823b4a2a8a9a30a41123b36a/roles/86d42fd805a14d5d8e83fbb0653bdb3e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:52.604648 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b51d5-c03f-4203-b677-9010c8349534 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.604935 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b51d5-c03f-4203-b677-9010c8349534 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.605213 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b51d5-c03f-4203-b677-9010c8349534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.624215 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ff2b51d5-c03f-4203-b677-9010c8349534 None None] MFA Rules not processed for user `a8e3d6ceca714fdb9b2c1a7b51c86f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:16:52.625417 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff2b51d5-c03f-4203-b677-9010c8349534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:52.664999 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:52.688276 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1472/2950] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:16:52.694676 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:52.696366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:52.740173 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:16:52.758893 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae25d5e-3870-4594-b247-33494f24f83e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:52.762772 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:52.763374 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:52.763658 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:52.763929 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:52.772902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:52.774378 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:52.784220 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1479/2951] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:16:56.760825 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:56.761383 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:56.761857 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:56.794342 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] MFA Rules not processed for user `1f681821823b4a2a8a9a30a41123b36a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:16:56.795676 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:16:56.866431 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1473/2952] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:16:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 16:16:56.870892 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:56.881808 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec4e0-22b1-4a2a-8286-5f198aa9f4fa service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:56.887692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:56.888337 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:56.888620 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:56.888898 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:56.889545 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:56.891840 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:56.921778 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1480/2953] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Sep 12 16:16:56 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 16:16:56.924997 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:56.937478 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b9884-72ec-4a52-a588-1413de593f48 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:56.999061 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:56.999577 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:56.999865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:57.000204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:57.001353 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:57.003338 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:57.025779 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1474/2954] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Sep 12 16:16:56 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 16:16:57.029762 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:57.040322 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bde4e-02a7-4033-8b7e-2ea02fcd8de8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:57.044989 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:57.045597 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:57.045902 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:57.046181 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:57.046826 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:57.048957 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:57.060487 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1481/2955] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 16:16:57.064970 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:57.076811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f572a-e24c-4255-81cd-97b0946fb682 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:57.082275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:57.083253 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:57.083638 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:57.083816 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] PATH_INFO: `/v3/projects/a2161507daf74156888752041105f63b/users/792613fef8da4d178b9456596717972b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:57.101280 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2161507daf74156888752041105f63b, user_id=792613fef8da4d178b9456596717972b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:57.104159 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:57.118906 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1475/2956] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:16:57 2022] PUT /identity/v3/projects/a2161507daf74156888752041105f63b/users/792613fef8da4d178b9456596717972b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:16:57.122903 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:57.133544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ff92-1526-4623-912f-80f15527cf6d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:57.194009 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:57.194623 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:57.194911 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:57.195190 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:57.195834 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:57.197441 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999d147-b223-499c-9a17-0675a58ca4fd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:57.208105 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1482/2957] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Sep 12 16:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 16:16:57.212344 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:16:57.224416 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74e055-ac2f-4500-8fa7-06a0967b8061 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:16:57.229348 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:16:57.229986 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:16:57.230268 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:16:57.230544 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] PATH_INFO: `/v3/projects/a2161507daf74156888752041105f63b/users/792613fef8da4d178b9456596717972b/roles/c939df765d074277be1dc85b4ec66f05` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:16:57.232467 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2161507daf74156888752041105f63b, user_id=792613fef8da4d178b9456596717972b, role_id=c939df765d074277be1dc85b4ec66f05)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:16:57.235829 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:16:57.246778 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1476/2958] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Sep 12 16:16:57 2022] PUT /identity/v3/projects/a2161507daf74156888752041105f63b/users/792613fef8da4d178b9456596717972b/roles/c939df765d074277be1dc85b4ec66f05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:17:01.927610 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bc017-535b-4d4d-9e63-559edda0fbb2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:17:01.928055 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bc017-535b-4d4d-9e63-559edda0fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:17:01.928298 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bc017-535b-4d4d-9e63-559edda0fbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:17:01.958839 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0b6bc017-535b-4d4d-9e63-559edda0fbb2 None None] MFA Rules not processed for user `792613fef8da4d178b9456596717972b`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:17:01.960108 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b6bc017-535b-4d4d-9e63-559edda0fbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:17:02.040266 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1483/2959] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:17:02.056744 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:17:02.058892 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:17:02.096167 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:17:02.143696 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9329-42c2-4e1a-bf54-e033be6e42ff admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:17:02.151588 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:17:02.152220 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:17:02.152498 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:17:02.152784 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:17:02.230230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:17:02.232001 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:17:02.245890 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1477/2960] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:17:12.036468 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:17:12.239150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=44 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:17:19.573510 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:17:19.574034 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:17:19.574129 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:17:19.592810 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] MFA Rules not processed for user `1f681821823b4a2a8a9a30a41123b36a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:17:19.594028 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:17:19.601203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1484/2961] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:17:29.587407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:17:35.894139 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:17:35.906316 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759eca9-bfbc-4424-9b44-4124dd075417 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:17:35.912664 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:17:35.913275 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:17:35.913561 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:17:35.913872 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:17:35.915624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:17:35.928508 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:17:35.930177 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:17:35.941511 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1478/2962] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:17:38.017685 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:17:38.073968 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b27dc-a643-4972-894f-63fadd36eddc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:17:38.077504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:17:38.078135 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:17:38.078425 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:17:38.078701 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:17:38.088500 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:17:38.089971 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13d531-4eb5-4ed4-9877-a80203ef1e21 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:17:38.099203 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1485/2963] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:17:45.936542 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:17:48.095057 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=26 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:18:52.968538 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:18:53.008382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5b037-924c-4c43-aa58-95a0e0d02102 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:18:53.012004 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:18:53.012616 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:18:53.012903 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:18:53.013180 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:18:53.014929 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:18:53.033817 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:18:53.035282 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:18:53.043435 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1479/2964] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:18:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:19:03.039667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=22 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:19:39.551555 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:19:39.552284 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:19:39.552567 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:19:39.574443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] MFA Rules not processed for user `1f681821823b4a2a8a9a30a41123b36a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:19:39.575822 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:19:39.597657 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1486/2965] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:19:39.909428 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:19:39.918667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1439c3-9c10-4a99-acb9-5733917c5f53 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:19:39.922270 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:19:39.922865 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:19:39.923150 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:19:39.923425 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:19:39.925138 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:19:39.935334 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:19:39.936746 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:19:39.944695 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1480/2966] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:19:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:19:40.377091 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:19:40.378522 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:19:40.455790 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11901069-a463-46c6-bb64-e08d128ead0c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:19:40.461626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:19:40.462256 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:19:40.462541 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:19:40.462814 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:19:40.464493 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:19:40.475942 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:19:40.477505 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:19:40.487501 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1487/2967] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:19:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:19:49.941035 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:19:50.483396 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=44 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:19:57.248849 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:19:57.257624 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e41e8-0eb0-4adc-bd2d-6100d2f2be6a service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:19:57.261385 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:19:57.262016 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:19:57.262296 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:19:57.262569 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:19:57.264269 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:19:57.299521 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:19:57.300880 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5781f5b-88ad-4011-949a-14b09ec43c39 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:19:57.320988 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1481/2968] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:19:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:20:07.318860 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=18 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:20:47.262990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:20:47.264536 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:20:47.278222 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b291a2f-9e6e-4ba2-b883-57b6658b700a service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:20:47.283366 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:20:47.283967 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:20:47.284250 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:20:47.284525 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:20:47.286243 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:20:47.298069 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:20:47.299663 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:20:47.310178 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1488/2969] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:20:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:20:57.305851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:21:38.074811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:21:38.075326 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:21:38.075436 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:21:38.094140 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] MFA Rules not processed for user `1f681821823b4a2a8a9a30a41123b36a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 16:21:38.095311 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:21:38.102036 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1482/2970] 10.222.0.20 () {58 vars in 958 bytes} [Mon Sep 12 16:21:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 16:21:41.039024 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:21:41.049052 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7e863-74b2-4113-9903-e7f841c507f1 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:21:41.052727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:21:41.053324 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:21:41.053606 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:21:41.053921 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:21:41.055607 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:21:41.069116 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:21:41.070581 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:21:41.082107 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1489/2971] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:21:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:21:44.578581 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:21:44.580263 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:21:44.598792 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5255-4615-4cd8-99f0-c092b1a5a4d4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:21:44.604277 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:21:44.604969 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:21:44.605346 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:21:44.605630 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:21:44.607430 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:21:44.620395 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:21:44.621982 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:21:44.634382 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1483/2972] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:21:51.077473 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:21:54.629853 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=9 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:23:31.146791 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:31.148318 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:23:31.161399 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d04099-3e77-4352-9a60-1deb114b7099 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:31.165727 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:31.166330 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:31.166615 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:31.166894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:31.175227 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:31.176648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:31.185990 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1490/2973] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:23:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 16:23:32.021725 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:32.037046 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba944-95a6-43e2-aea2-a4833c7718f6 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:32.043271 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c615724d4e5944169ead59e30e0d04b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:32.044169 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:32.044685 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:32.045188 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:32.047333 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:23:32.063076 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:32.063813 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:32.076876 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1484/2974] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:23:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:23:41.182058 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:23:42.071047 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:23:44.534542 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:44.543882 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24617-44c9-498e-8467-882d5fe96144 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:44.547585 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0621ba7d8a164789958d1cadba19ea6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:44.548118 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:44.548400 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:44.548673 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:44.609648 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:44.611102 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:44.620491 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1491/2975] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:23:47.888667 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:47.901011 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937566-cf5e-4a8b-b6a0-854e070b903a service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:47.906468 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:47.907074 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:47.907362 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:47.907645 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:47.909421 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:23:47.921809 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:47.923477 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:47.934439 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1485/2976] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:23:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:23:50.830443 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:50.841985 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf56ed-351a-4197-977a-5bbd3a03bbd2 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:50.846904 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:50.847569 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:50.847850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:50.848125 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] PATH_INFO: `/v3/users/1f681821823b4a2a8a9a30a41123b36a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:50.849237 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f681821823b4a2a8a9a30a41123b36a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:50.851199 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:50.897422 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f681821823b4a2a8a9a30a41123b36a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:50.898094 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f681821823b4a2a8a9a30a41123b36a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:50.898692 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f681821823b4a2a8a9a30a41123b36a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:50.905513 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f681821823b4a2a8a9a30a41123b36a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:50.918098 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f681821823b4a2a8a9a30a41123b36a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:50.932725 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1492/2977] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:23:50 2022] DELETE /identity/v3/users/1f681821823b4a2a8a9a30a41123b36a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:23:51.103811 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:51.114761 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e16e3-b109-4ad8-99c7-f8bd1274b964 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:51.172323 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:51.172958 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:51.173242 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:51.173517 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] PATH_INFO: `/v3/projects/b0664ad8cda546e59bcadd62db443bfc` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:51.174712 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0664ad8cda546e59bcadd62db443bfc)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:51.176710 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:51.227687 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0664ad8cda546e59bcadd62db443bfc', 'request_id': 'req-94e424d2-084c-44ac-a0c5-8f01edb42703'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.233915 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0664ad8cda546e59bcadd62db443bfc', 'request_id': 'req-94e424d2-084c-44ac-a0c5-8f01edb42703'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.239653 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0664ad8cda546e59bcadd62db443bfc', 'request_id': 'req-94e424d2-084c-44ac-a0c5-8f01edb42703'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.252273 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] The token cache is being invalidate because project b0664ad8cda546e59bcadd62db443bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:23:51.252573 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.253204 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.254767 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1486/2978] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:23:51 2022] DELETE /identity/v3/projects/b0664ad8cda546e59bcadd62db443bfc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:23:51.258781 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:51.260193 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:23:51.273370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b64804-5ae0-428d-ba86-869f14376c4a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:51.332851 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:51.333485 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:51.333804 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:51.334083 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] PATH_INFO: `/v3/users/792613fef8da4d178b9456596717972b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:51.335214 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=792613fef8da4d178b9456596717972b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:51.337128 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:51.342504 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:23:51.392348 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792613fef8da4d178b9456596717972b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.393005 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792613fef8da4d178b9456596717972b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.393626 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792613fef8da4d178b9456596717972b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.401142 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792613fef8da4d178b9456596717972b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.413544 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792613fef8da4d178b9456596717972b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.426915 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1493/2979] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Sep 12 16:23:51 2022] DELETE /identity/v3/users/792613fef8da4d178b9456596717972b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:23:51.596747 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:23:51.598243 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:23:51.613102 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e424d2-084c-44ac-a0c5-8f01edb42703 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:23:51.668873 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8e3d6ceca714fdb9b2c1a7b51c86f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45e559d5550544c28dd39b8ed456ebde', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:23:51.669528 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:23:51.669819 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:23:51.670093 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] PATH_INFO: `/v3/projects/a2161507daf74156888752041105f63b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:23:51.671203 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2161507daf74156888752041105f63b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:23:51.673092 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:23:51.694091 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:23:51.720325 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2161507daf74156888752041105f63b', 'request_id': 'req-dce30818-a81b-4811-aa98-fdec40541f26'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.726513 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2161507daf74156888752041105f63b', 'request_id': 'req-dce30818-a81b-4811-aa98-fdec40541f26'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.732087 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2161507daf74156888752041105f63b', 'request_id': 'req-dce30818-a81b-4811-aa98-fdec40541f26'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.744292 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] The token cache is being invalidate because project a2161507daf74156888752041105f63b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 16:23:51.744587 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.745209 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dce30818-a81b-4811-aa98-fdec40541f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 16:23:51.746707 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1487/2980] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Sep 12 16:23:51 2022] DELETE /identity/v3/projects/a2161507daf74156888752041105f63b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 16:24:01.423407 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:24:01.741230 dv-cinder-851996-10-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 16:25:53.216092 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 16:25:53.217494 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:25:53.305907 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad79d44-9c8a-47cc-9166-a5b307f4c3fa admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 16:25:53.364962 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45b541633f92420e9b118c25775d646a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92831fcbcbee481387cf1e4c50cbb32e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 16:25:53.365579 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 16:25:53.365894 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 16:25:53.366179 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 16:25:53.368436 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 16:25:53.380233 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 16:25:53.381850 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dba8c-c4db-4370-a50d-bce582481302 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 16:25:53.392370 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1494/2981] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Sep 12 16:25:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 16:26:03.388045 dv-cinder-851996-10-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=67 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}